Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amazon's Policy Violation #56

Open
jose777js opened this issue Jan 26, 2022 · 7 comments
Open

Amazon's Policy Violation #56

jose777js opened this issue Jan 26, 2022 · 7 comments

Comments

@jose777js
Copy link

Does panda Crazy violate this Amazon usage policy?
I mean this policy: do not disrupt or harm the operation of the site or the integrity of the market (eg scripts and automation tools that call MTurk continuously and at high frequency are not OK). As a result, we will have to make hard calls from time to time when we notice unusual activity on the account resulting from a specific automation tool or script. Remember that you are responsible for any script or automation tool you use with MTurk, and your use of an automation script or tool is at your own risk.

@JohnnyRS
Copy link
Owner

By default the timers are all set to a time that should not cause any PRE's. Panda timer is at 1 second, search timer is at 950ms and queue timer is at 2 seconds. It seems Amazon has different PRE settings for those 3 pages. If Amazon notices a high frequency lookup they send a PRE. The script shows how many PRE's it gets on the panda page and the search page. The script has a built in limit for any timer and won't let the user set the timer below 700ms.

Like Amazon has said, the user is responsible to make sure they don't get a lot of PRE's which is Amazon's first warning of a problem. You can get a lot of PRE's if you are using another script at the same time or lowered the timers too low. A few PRE's should be OK but if you are getting hundreds then that could be a problem. Raise the timers until you don't get any more PRE's. I don't know at what point Amazon would find a problem so keep those PRE's at 0 as much as you can.

@jose777js
Copy link
Author

jose777js commented Jan 26, 2022 via email

@JohnnyRS
Copy link
Owner

I would say it is very unlikely that the script wouldn't detect a PRE especially if Amazon is sending a lot of them. If using two scripts then there is a chance the other script is getting more PRE's than this script so the number wouldn't be accurate but it should be showing some at the top.

@jose777js
Copy link
Author

jose777js commented Jan 27, 2022 via email

@JohnnyRS
Copy link
Owner

Amazon hasn't defined what would be called a high frequency that I know of. The PRE is our best way of knowing if it's too fast or fine. From my observations, panda calls get no PRE's at 1 second and search page calls get no PRE's at 1 second too. That is the default I set in the script. You may find the search page timer could be lower than 1 second though. Think about PRE as a warning but not as a violation. Getting over 1000 PRE's a day probably would be extreme but a few hundred a day may not be. Amazon has not said how many PRE's a day would be a violation so best to not get any PRE's and not worry about it.

@jose777js
Copy link
Author

jose777js commented Jan 28, 2022 via email

@mm2048
Copy link

mm2048 commented Jan 28, 2022

Amazon does not offer an API interface for workers. The documentation does not list throttle rates that trigger a slow down request (HTTP 429 response). My guess is that the AWS servers determine a value based on their server load.

I keep a steady 24/7 and have not had issues. I will notice 15-50 429 responses come up in a short 5-10 minute period. But then nothing for another week.

My interpretation of the AUP has always been that Amazon is looking to prevent workers from purposely flooding the network to prevent other workers from grabbing HITs or from requesters from getting their batches done.

I do not really see it as being an issue with PMAX for two reasons.

  1. Amazon owns the AWS infrastructure. Bandwidth is not expensive.
  2. Customers come to AWS specifically because of the integration with MTurk workers. Request loads are a good measure of worker activity.

Who came up with labeling them as a PRE response and not a 429?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants