Unfortunately, there's no bright line test. The inquiry is highly fact specific and often reasonable minds can differ on the analysis.
You can apply some common sense, though. If you are creating a jersey with the number 4 and the name "Brett Favre", are you intending to convey to consumers that this is a replica/reproduction of a Brett Favre jersey from one of his teams? A pink jersey with "Favre" and the number 4 might be ok, while a the same jersey in yellow and green might get you in trouble.