Question
I want my app to start working with digital ocean and to query around 10k api's in 10-30seconds
- Posted on November 18, 2021
- Cloud ComputingAPIKubernetes
Asked by benshalev849
I have an app that queries a lot of APIs, it relies on speed and it needs to be super fast.
I want to start running all the processing on the cloud instead of the client-side to make the app faster but I don’t know what will be the best solution.
Atm, it’s run in JS meaning it’s all run client-side so it’s very slow. Some APIs can take 30seconds (very good) and some can take 3-5mins.
My goal is to take most APIs and query them in a range of 10-30seconds or so (If the API is good to semi-good).
After the API requests are received there is very little parsing/calculating so that isn’t heavy and then the data should be sent to a database.
What will be the best solution to get those API queries as fast as possible? I thought of these options: Multiple very small computes. 1-2 big computes Managed Kubernetes (Kinda complicated since not very familiar with it) Computes w/ docker (Kinda complicated since not very familiar with it).
Of course, I will need to also buy proxy servers to route the requests through in case of rate-limiting on the server-side.
Ty very much for helping :)
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.
Hello,
I believe that all suggestions have their benefits. What I would personally do is to do it step by step, so first create a backend service that is capable of consuming the APIs and testing it out on a Droplet directly.
You can take a look at this video on how to deploy a Node.js application on an Ubuntu server:
Once you have the backend service ready, you can take it a step further and dockerize it as per the instructions here:
https://www.digitalocean.com/community/tutorials/how-to-build-a-node-js-application-with-docker
That way you will be able to easily start multiple instances of your Node.js app on the same server.
As you mentioned, the next step would be to go for a container orchestration like Kubernetes. Here are some useful resources:
Hope that this helps! Best, Bobby