High CPU usage by AttackMapServer.py

[ChristophRh]

Today I recognized a very high CPU usage by the AttackMapServer.py which was not the case a week ago.
Anyone else having issues?

There was an update of the geoip_attack_map repository a few days ago. Could this update, which changed important code parts, be the reason for the high CPU usage or is it just an issue on my side?

Thanks for your feedback!

[t3chn0m4g3]

Can you give more info on your setup, hardware, average events per day, connected clients, etc.?

It should be noted however that the AttackMapServer works differently now. Except for MapBox all code is served locally, with aiohttp, aioredis and asyncio to ensure better performance. Also I activated gzip and brotli support in NGINX and deferred the loading of JS code which will result in lots of parallel connections on page load.

Should you spot errors in the code, the setup or while debugging please share.

[ChristophRh]

Works perfectly, thanks a lot for the quick fix!

Any idea where these wrong values of my geoip_ext.ip originate from?
Should I open a new topic for this issue?

[t3chn0m4g3]

Great, thanks for the swift feedback!

Any idea where these wrong values of my geoip_ext.ip originate from?

Please check the Kibana Dashboard > Discover and search for some events. Enlarge the events and check its fields for geoip_ext.ip and compare it to the t-pot_ip_ext field. Typically these offer the same IP (identical to /etc/issue).

Since it is identical to the one from Kibana, as you mentioned earlier, it is the correct behaviour and necessary to ensure dynamic mapping of T-Pots in distributed scenarios.

[ChristophRh]

geoip_ext.ip and t-pot_ip_ext are identical and the same as the public IP listet in /etc/issue.
But still geoip_ext.latitude and geoip_ext.longitude are pointing to Seattle which makes no sense. If I lookup the geolocation of my IP address it is in Germany and that should be correct since I'm using an AWS instance in Frankfurt.

I know that this issue is not related to your attack map anymore, but I thought you could give me a quick hint where these wrong values come from. But I can also start a new discussion for this topic.

[t3chn0m4g3]

This is geopoint related. For the old map the local geopoint was determined from ipinfo (as seen here) while the new one uses geo point from Logstash (which uses GeoLite); ipinfo had too many disadvantages (i.e. only one T-Pot supported) and will be removed entirely. As long as Logstash does not receive a GeoLite DB update the info on your end will be not the same as before. You can however use volumes and exchange the GeoLite database on your own.

[ChristophRh]

I managed to do that :)
Thanks a lot for your help!

[bob3rocks]

Can you tell me what the fix was for this? I've got the same issue, had to disable AttackMapServer in the tpot config due to sustained 90%+ CPU. I installed T-Pot 2.3 two weeks ago, and it was working great for the first few days. Thanks

[t3chn0m4g3]

The new image should have downloaded already, if not just follow the steps outlined above. The new image solves the issue.

[bob3rocks]

Gotcha! Thanks, I will try that

…

On Sun, Jan 15, 2023 at 1:17 PM Marco Ochse ***@***.***> wrote: The new image should have downloaded already, if not just follow the steps outlined above. The new image solves the issue. — Reply to this email directly, view it on GitHub <#1241 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFPYFARME42ZLAVV2HENCDTWSRSP3ANCNFSM6AAAAAATVR2ASM> . You are receiving this because you commented.Message ID: ***@***.*** com>

-- Bob Wachunas Breakaway Funk <http://breakawayfunk.com/> 707-536-1182

[bob3rocks]

Looks good now! Thanks

…

On Sun, Jan 15, 2023 at 1:17 PM Marco Ochse ***@***.***> wrote: The new image should have downloaded already, if not just follow the steps outlined above. The new image solves the issue. — Reply to this email directly, view it on GitHub <#1241 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFPYFARME42ZLAVV2HENCDTWSRSP3ANCNFSM6AAAAAATVR2ASM> . You are receiving this because you commented.Message ID: ***@***.*** com>

-- Bob Wachunas Breakaway Funk <http://breakawayfunk.com/> 707-536-1182

[t3chn0m4g3]

Deployed 1.1.0 yesterday. Let me know what you think.

[ChristophRh]

It failed to load resource PS.png, which should be the flag of Palestine I guess.
And today the map did not load any attacks, but after a restart of tpot it works again. I will keep an eye on that.

Everything else looks great, thank you!
But is there a way to prevent the attack map from updating automatically so that I can choose when to update?

[t3chn0m4g3]

Thanks, I will be updating all flags with Flagpack shortly.

Check the Readme for details on updates and daily reboot.

[ChristophRh]

The Readme does not contain enough details for me. I just want to exclude the map from automatic updates.
Can I do that by removing the line image: "dtagdevsec/map:2204" in /opt/tpot/etc/tpot.yml or does that break something.
I'm not experienced with docker.

[t3chn0m4g3]

It is either all or nothing (/etc/crontab), removing the image line will result in an error and T-Pot will not start.

[ChristophRh]

okay
thank you very much for your help!