Very restrictive usage limit on the overpass api since a while?

Question

My Android app BikeNode was successfully using the overpass api for years. I was using this format:

http://www.overpass-api.de/api/xapi?node[rcn_ref=*][bbox=5.170799818181818,51.363934891891894,5.534436181818181,51.580151108108105]

But since a while I face issues. Now often I get this error:

runtime error: open64: 0 Success /osm3s_v0.7.53_osm_base Dispatcher_Client::request_read_and_idx::rate_limited. Please check /api/status for the quota of your IP address.

Before August-2016 I never received this error. Usage of my BikeNode app did not increase. If I check this /api/status page I see a message like this:

Connected as: 2441151942
Rate limit: 2
Slot available after: 2016-10-06T10:34:46Z
Slot available after: 2016-10-06T10:36:35Z
Currently running queries (pid, space limit, time limit, start time):

I tried to switch to the Rambler Overpass API Instance, using this format:

http://overpass.osm.rambler.ru/cgi/xapi?node[rcn_ref=*][bbox=5.170799818181818,51.363934891891894,5.534436181818181,51.580151108108105]

But this regularly shows this error:

runtime error: open64: 0 No error: 0 /osm3s_v0.7.52_osm_base Dispatcher_Client::request_read_and_idx::timeout. Probably the server is overcrowded.

If I check its /api/status page, I see a message like this:

Connected as: 2130706433
Rate limit: 0
Currently running queries (pid, space limit, time limit, start time):
75742   536870912   180 2016-10-06T10:42:52Z
75784   536870912   180 2016-10-06T10:43:00Z
75770   536870912   180 2016-10-06T10:42:57Z
75796   536870912   180 2016-10-06T10:43:02Z
75797   536870912   180 2016-10-06T10:43:02Z
75808   536870912   180 2016-10-06T10:43:02Z
75809   536870912   180 2016-10-06T10:43:02Z

The general Overpass API/status page explains about an issue since 23-aug-2016.

Does anybody know what's happening here? What is the current status of the servers? Is there a limit configured since some weeks? Are the servers overloaded since some weeks? Does anybody have tips/hints?

PS1. Both servers (Main and Rambler) have the following Usage Policy:

Both servers have a total capacity of about 1.000.000 requests per day. You can safely assume that you don't disturb other users when you do less than 10.000 queries per day or download less than 5 GB data per day.

PS2. Related question: https://help.openstreetmap.org/questions/52333

Answer 1

6

The "xapi" endpoint has been severely limited to counteract a malicious or extremely thoughtless user/app developer making between 300,000 and 500,000 requests every hour since the 20th of August. If you were to switch to "Overpass QL" or XML queries instead of using the "xapi" endpoint you'd still get responses.

permanent link

answered 06 Oct '16, 21:29

Frederik Ramm ♦
73.3k●86●664●1137
accept rate: 24%

1

I'm switching to "Overpass QL" and that works. Thanks for the input! I'm using this format now:

http://www.overpass-api.de/api/interpreter?data=node["rcn_ref"](51.42283,5.44612,51.45386,5.50655);out;

(07 Oct '16, 21:18) alexvanderli...

1

I skipped to conclusions to fast. The "Overpass QL" format shows the same availability issues. After testing some time, this seems to be the behavior for both end-points (.de/api/xapi and .de/api/interpreter):

From one IP address you can use it twice in a row. After this you'll have to wait some minutes before you can use it again.
The performance from phone (from my android app) is very slow (around 15s response time).

(09 Oct '16, 20:36) alexvanderli...

1

This related issue at the server has been registered at 7-oct-2016: https://github.com/drolbr/Overpass-API/issues/323

(10 Oct '16, 21:15) alexvanderli...

1

This issue is fixed: https://github.com/drolbr/Overpass-API/issues/323 . As a result the server response is like it used to be. Thanks a lot for fixing this issue!

(12 Oct '16, 18:54) alexvanderli...

Very restrictive usage limit on the overpass api since a while?

Related questions