Portugal would largely be unaffected. French and English colonies would largely also be unaffected, but would be much slower in conquering people. The biggest impacts are with Sweden and Poland actually, and of course the native American peoples at the time.