Having an issues with recent pulls or db update giving db errors on game server

[Snugglesffxi]

Low Priority, as this is just a test box (unless the issues turn out to be global)

On July 17, when I pulled from base branch and updated with dbtool/rebuilt, the game client kept throwing a lot of DB errors:

Pastebin: Game Server Log

Previous working merge/sql update before that was June 27.

The server is up to date as of writing this, and the only conflicts were the conf files, as pointed out to me by clay.

conf/default/login.conf: needs merge
conf/default/server_message.conf: needs merge
conf/default/version.conf: needs merge

I resolved the conflicts and relaunched the game to the same errors but didn't want to pester Clay anymore so I sat on it for a few days.

Today, I was able to restore a backup of the db using dbtool and got the server up and running, but due to DB being behind its causing other issues (obv). So I pulled again, used dbtool to update, rebuilt, and got the same db errors as the other day.

Clay mentioned with all the recent cor changes and DB formats theres bound to be issues... I'm apprehensive to even post this because I know there's going to be growing pains but at the same time I wanted to know if it's just me or a global issue and if it is the latter to make sure its a known issue.

[claywar]

You default confs shouldn't have an impact on the server, since they need to be copied one level higher. What could be going wrong is that its expecting some value that's not being set in the updated version of the configuration.

There's a PR open that changes one item, but the last time I can think of it changing was minor, when updating some default strings, and before that when adding capacity point modifier. There have been a ton of DB changes recently, so that would explain the errors on startup, but what is dbtool doing?

[Snugglesffxi]

DBtool ran fine (to my eyes, idk)

Database saved!
Importing accounts_sessions.sql...
Importing augments.sql...
Importing automaton_abilities.sql...
Importing instance_entities.sql...
Importing instance_list.sql...
Importing item_equipment.sql...
Importing item_latents.sql...
Importing item_mods.sql...
Importing item_mods_pet.sql...
Importing mob_family_system.sql...
Importing mob_groups.sql...
Importing mob_pools.sql...
Importing mob_resistances.sql...
Importing mob_skill_lists.sql...
Importing mob_skills.sql...
Importing mob_spawn_mods.sql...
Importing mob_spawn_points.sql...
Importing mob_spell_lists.sql...
Importing npc_list.sql...
Importing spell_list.sql...
Importing zone_settings.sql...
Importing zone_weather.sql...
Finished importing!
Checking migrations...
Running migrations for Search comment and language columns...
Finished migrations!
Closing connection...

[claywar]

Hmm, could you post the beginning of map logs on start? Don't need the full output, since I'm assuming its error spam, just need to see a page or so of the errors to try to figure out what table you might be missing.

[Snugglesffxi]

[19/Jul] [03:57:21] [Info] Topaz
[19/Jul] [03:57:21] [Status] do_init: begin server initialization...     - [OK]
[19/Jul] [03:57:21] [Status] do_init: map_config is reading              - [OK]
[19/Jul] [03:57:21] [Status] luautils::init:lua initializing...[19/Jul] [03:57:21] [Error] ROEUtils: Unknown flag hidden for record #4085.               - [OK]
[19/Jul] [03:57:21] [Status] do_init: sqlhandle is allocating            - [OK]
[19/Jul] [03:57:21] [Status] do_init: zlib is reading                    - [OK]
[19/Jul] [03:57:21] [Status] do_init: loading items                      - [OK]
[19/Jul] [03:57:24] [Status] do_init: loading plants                     - [OK]
[19/Jul] [03:57:24] [Status] do_init: loading spells[19/Jul] [03:57:24] [Error] ROEUtils: Unknown flag hidden for record #4085.                  - [OK]
[19/Jul] [03:57:25] [Debug] UpdateGuildPointsPattern is finished. New pattern: 4
[19/Jul] [03:57:26] [SQL] DB error - Unknown column 'systemid' in 'field list'

is this what you need?

[Snugglesffxi]

oops maybe this?

[19/Jul] [03:57:36] [Debug] InitializeWeather Finished
[19/Jul] [03:57:36] [Status] The map-server is ready to work...
=======================================================================
[19/Jul] [03:57:36] [Debug] [Lua] Garbage Collected (Step)
[19/Jul] [03:57:36] [Debug] [Lua] Current State Top: 0, Total Memory Used: 17603kb
[19/Jul] [03:57:38] [Warning] luautils::GetMobByID Mob doesn't exist (17220000)
[19/Jul] [03:57:38] [Warning] luautils::GetMobByID Mob doesn't exist (17220001)
[19/Jul] [03:57:38] [Warning] luautils::GetMobByID Mob doesn't exist (17260613)
[19/Jul] [03:57:38] [Warning] luautils::GetMobByID Mob doesn't exist (17559558)
[19/Jul] [03:57:38] [Error] luautils::onGameHour: ./scripts/zones/Dangruf_Wadi/Zone.lua:67: attempt to index local 'nm' (a nil value)
stack traceback:
        ./scripts/zones/Dangruf_Wadi/Zone.lua:67: in function <./scripts/zones/Dangruf_Wadi/Zone.lua:65>```

[claywar]

You might be behind on some other core updates as well. The hidden flag for RoE was added some time ago, and that's one place you're erroring out (#392 ), systemid is from Teo's rework for SDT, and that table should have been pulled in, assuming you've pulled latest.

Can you try pulling the latest code in a separate location to see if the errors are still occurring on your end?

[Snugglesffxi]

okay let me try that

[Snugglesffxi]

[19/Jul] [04:49:22] [Error] File not found: ./conf/packet_udp.conf
[19/Jul] [04:49:22] [Status] do_init: begin server initialization...[19/Jul] [04:49:22] [Error] Could not read English server message from: ./conf/server_message.conf
         - [OK]
[19/Jul] [04:49:22] [Status] do_init: map_config is reading              - [OK]
[19/Jul] [04:49:22] [Status] luautils::init:lua initializing...          - [OK]
[19/Jul] [04:49:22] [Status] do_init: sqlhandle is allocating            - [OK]
[19/Jul] [04:49:22] [Status] do_init: zlib is reading                    - [OK]
[19/Jul] [04:49:22] [Status] do_init: loading items                      - [OK]
[19/Jul] [04:49:36] [Status] do_init: loading plants                     - [OK]
[19/Jul] [04:49:36] [Status] do_init: loading spells                     - [OK]
[19/Jul] [04:49:40] [Debug] UpdateGuildPointsPattern is finished. New pattern: 5
[19/Jul] [04:49:51] [Status] do_init: loading zones                      - [OK]
[19/Jul] [04:51:21] [Status] do_init: server is binding with port 54230  - [OK]
[19/Jul] [04:51:21] [Debug] InitializeWeather Finished
[19/Jul] [04:51:21] [Status] The map-server is ready to work...
=======================================================================
[19/Jul] [04:51:21] [Debug] [Lua] Garbage Collected (Step)
[19/Jul] [04:51:21] [Debug] [Lua] Current State Top: 0, Total Memory Used: 31657kb

[19/Jul] [04:49:04] [Error] File not found: ./conf/packet_tcp.conf
[19/Jul] [04:49:04] [Info] Console Silent Setting: 0
[19/Jul] [04:49:04] [Error] version info configuration file not found at: conf/version.conf
[19/Jul] [04:49:04] [Error] maint configuration file not found at: conf/maint.conf
[19/Jul] [04:49:04] [Status] The login-server-auth is ready (Server is listening on the port 54231).

[19/Jul] [04:49:04] [Status] The login-server-lobbydata is ready (Server is listening on the port 54230).

[19/Jul] [04:49:04] [Status] The login-server-lobbyview is ready (Server is listening on the port 54001).

[19/Jul] [04:49:04] [Status] The login-server is ready to work...
[19/Jul] [04:49:04] [Status] Console input thread is ready..

yup, fresh install, its an improvement i guess..

1. why is git ignoring some updates? and then telling you you're up-to-date when you really aren't.
2. why is new installs lacking those conf pictured in errors, dbtool also had issues with version.conf

Importing zonelines.sql...
Importing triggers.sql...
Finished importing!
Update client version? [y/N] y
Error writing version.

im assuming all this is more to do with git than anything here tbh, or user error LOL

[claywar]

To try to answer some of your questions:

1. Not sure why you wouldn't be seeing git errors on merge conflicts. I use a separate client to handle things, and haven't seen that one before. Are you sure you're targeting the right repo and branch?
2. Conf files are not in their intended locations by design. When configuring a new server, you need to copy them the the conf directory from conf/default

[Snugglesffxi]

Not sure why you wouldn't be seeing git errors on merge conflicts. I use a separate client to handle things, and haven't seen that one before. Are you sure you're targeting the right repo and branch?

yes! remote base branch from gitbash (never had this issue before)

Conf files are not in their intended locations by design. When configuring a new server, you need to copy them the the conf directory from conf/default

omfg i forgot the other files outside of the 3 you need to edit!! 🥟