Am I in over my head?
So I've got a live production site built with Bubble that allows my community to submit deals they find on the web within my niche and upvote them. To really scale my vision of this site, I want to built it more into a price comparison engine, think Trivago but for PC tech. The problem is I have spent endless nights trying to find the right solution to make this happen and I feel like Xano may be last hope.
What I'm Trying to BuildThe bread and butter of any price comparison site is obviously pulling in data (regularly) from various APIs and formating that mess into one easy-to-read database.
My plan is to start by indexing around 10,000 products and ultimately scale up to around 40,000. Starting out, each product will display the price from three different merchants (Amazon, Best Buy & Newegg) and then I'll slowly add more.
Searching 40,000 database entries in Bubble is a breeze, but running workflows to check the various external API feeds even just once a day is where the Bubble site can slow down quick. Workflows become messy and even if I mastered 'recursive workflows' (which I seem to struggle with) the math on checking even 10k products x 3 merchants every single day tells me my site is basically going to need workflows running 24/7 that could easily break at anytime. Not good.
But the general UI/UX of the site will be simple: You're looking to buy a gaming monitor? Here's an up-to-date list of the best monitors at the best price right now. And how do I avoid duplicate products from these various merchants? Every product has a 'UPC' number which is unique to that product and used across all merchants.Â
I Almost Settled on Make/IntegromatI'm a visual learner and Make.com catered very well to that, and the pricing is reasonable as well. I was willing to pay the extra cash to have an easy-to-use workflow that could pull in all this messy data, organize it nicely in a Google Spreadsheet, and then sync that spreadsheet to my Bubble database. The problem? While cleaner, my Bubble site would still need to be constantlely running a backend workflow to pull in all the data thefore slowing the site. Not to mention the # of 'operations' involved with my large database would drive my Make subscription costs through the roof! There's got to be a better way.
Enter XanoAnytime I brought up working with large datasets that need to be updated regularly on the Bubble forums I'd always have at least one person say 'why not just put all the data on an external DB and then just call the API every time? The problem was that these other DB solutions didn't have 'nocode' methods of connecting to API's to retreieve data like Bubble had. Plus, even if I could manage to get all the data in one-place, having an API call to that database everytime a user goes to my site and every time they change the search criteria/load a new page, what am I going to do when even 100 people are on my site at once looking at products? Every single provider suddenly get expensive.
Except Xano which by some miracle has no rate limiting on an affordable plan and no record limit?! I was sold.
The problem? I'm so burnt out of taking a week to learn a platform only to find out it's not the right solution. It seems like from my research that having an external database is the best solution for my type of site. While I wish I could 'be up in running within minutes' I feel like to build out all the API logic on Xano between learning everything is going to take weeks.
Is Xano the best approach for this? If I have say 200 people on my site at the same time all constantely pulling data from my Xano database, is this going to scale with that? Are there complexities/costs that I'm not aware of? My plan/hope is to accomplish this with the 'Launch' plan.
/rantÂ
Other
4 replies