Honestly, unless you are dead-set on vinyl, it might be worth it to just consider an all streaming set-up. Vinyl is expensive and gear can be as well. Digital overall has been sound quality, especially newer mixed albums. That said, if you are set on going down the rabbit hole, here's some gear I've read good things about as a starting point. I'd encourage you to seek out reviews on any specific gear. Headphones are a good alternate to speakers, especially if you live in an apartment or have cohabitants that prefer quietude. That's another rabbit hole, so let me know if that is more to your interest.
Best bang for buck component is always going start with the speakers. You can debate gear importance beyond that. There's no harm in starting small (and spending the rest of your budget on records). I would advise to find a set of speakers and work from there on the rest of the components. The biggest thing to look for is whether the speakers are powered/active (no external amplification required) or passive (requiring external amplification). Nothing wrong with active speakers, especially to start. The biggest part of getting passive speakers IMO is the ability to swap in and out amplifiers as needed. I am hoping my current speakers outlive me so I have an external amp on both my desktop and living room set ups.
So some food for thought/homework: Do you want active or passive speakers? What sort of creature comforts do you want? What sources will you plan to use (streaming, vinyl, CD, etc.)?
Speakers:
- $120: Edifier R1280T -
Amazon
Budget option. No external amplification required.
- $300:
Amazon
Note these are not currently on sale but often go for $300. Check
camelcamelcamel to gauge frequency. I use these at my desk and I adore them. They will require amplification.
Turntables:
- $150: AT LP60 -
Amazon
Beginner table with phono preamp included. Not really upgradable.
- $350: AT LP120 -
Amazon
This is what I have. Beginner range for upgradable turntables. Anything with a counterweight is a good starting point. This TT can support many different cartridges, has fast speed adjustment between 33, 45, and 72 rpm. Also includes a phono preamp.
Amplifiers:
Only needed if you get passive speakers.
If space isn't a concern and if it's going to hook up to your TV, I'd recommend getting an integrated amplifier. This will most likely add in a lot of nice to hav features like Spotify Connect, Chromecast, Airplay, etc. Some also come with phono stages, if you do opt for a record player. If your turntable comes with a phono stage like the two mentioned above, just think of it as future proofing in case you upgrade to a nicer TT.
Here are some videos on the subject. I would happily recommend watching additional content from any of these channels.
This guy mostly focuses on desktop set-ups & headphones, but still a good resource if you want to explore.