I adore worldbuilding. Even though it can (and really should) require a good bit of research, I consider worldbuilding one of the most fun aspects of writing in a genre like fantasy and science fiction (maybe even horror). You mean I get to make up all kinds of fun stuff? Awesome!
I think that most people who write in those genres enjoy worldbuilding (or they’d write in a different genre), which is good since people who read those genres enjoy the results of the worldbuilding. It’s part of what pulls you in – the richness of the details and how real the world feels, especially. The Harry Potter series would not have been so popular without moving staircases, butter beer, and Quidditch. Similarly, Ender’s Game would be a different book without Thirds, the Command School, or a Molecular Disruption Device.
Since there are only so many possible plots, worldbuilding is a great way to make your story more interesting and unique. So go for it! If you have a great idea for a kind of magic or time travel or mutant creature, put time and thought into the details that make it feel real and believable. Just don’t get so caught up that other parts of the story don’t get enough attention.
The more aspects of the world that seem solid to the reader, the more real the world will feel, so you want the world details to be well rounded. Yes, you need to figure out the magic or cool technology that you’re really excited about. At the same time, the characters need to eat something. They need to sleep somewhere. They need a way to buy stuff, they need something to wear, and they may even need protection from nature (weather, animals, etc). Don’t get so caught up in the big stuff that you forget to think about all the day-to-day stuff.
And it is ok to have some of it be like the real world. You don’t have to make everything different. In fact, the more similar it is to the real world, the easier it will be for readers to absorb and enjoy. How much needs to be different depends on the world you’re creating. The balance between common and unique will vary from book to book and author to author. There’s plenty of room for flexibility.
So have fun! Be creative and make up all kinds of cool and interesting details for the world. Then, weave them into the plot and characters until you pull us in. Fantasy and sci fi readers love to dig their teeth into a good world, so give them something to chew on!