Our #1 job here at Keen IO is to maintain the reliability and stability of our API. As you might expect, we’ve put a number of limits and protections in place to safeguard the service and make sure existing customers are protected.
Today, we had a couple questions come up about the forms of limiting we’ve employed over the last couple weeks. In all the hubub we may’ve lacked a clear explanation of each, so I’ll give one now. We also updated our documentation related to each of these today. If anything below is unclear, shoot us a question!
Rate Limits
Keen’s had this for a long time. They limit the number of queries a customer can perform within a 60 second window. These limits are enforced at the project level. These limits are pretty high at about 1,000 a minute.
You can see these limits in our docs here.
Note: Rate limiting is soon going to be improved significantly. This may change our limits. We’ll talk about that more when the time comes.
Concurrency Limits
This is a new limit, and it’s enforced per-Organization and has two parts: Only ~24 queries are allowed to be executing simultaneously in each data center Only ~24 queries are allowed to be pending execution in each data center
These limits are documented here.
Concurrency limits are both new and pretty severe right now. We’d love to raise them in the future when we can handle it.
Fast Failure
This isn’t a rate limit per se. It’s a combined guarantee and protection feature. If we haven’t been able to begin executing your query within 10 seconds, we will send it back to you with a failure. This feature is going to get an improvement tomorrow to make it work more like what I just said above. Today it’s allowing some queries to hang around a bit longer.
Fast failure is documented here.
Why Do These Limits Exist Now?
For a number of reasons around performance and system health, these are the limits we’ve put in place to keep averages reasonable given our ability to execute queries. We’d like them to be higher, but they are what they are until we can improve performance within the stack.
Long story short: As new customers stretch the platform new ways, we want to make sure existing customers are protected. We’ve grown really fast and we’re having trouble keeping up. We’ve been working super hard over the last week to get things to a stable place. Now we can start aiming for making it faster. 🙂
If you have any questions about rate limits, please reach out to us!
