libcurl during build missing

[belamu]

During make I get the following error:

make[2]: *** No rule to make target '<parentdir>/ceema/3rdparty/curl/lib/libcurl.so', needed by 'libceema.so.0.0.1'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:118: src/CMakeFiles/ceema.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

I have installed: sudo dnf install libcurl-devel libcurl
I am confused why make needs something from /3rdparty/curl since this directory or submodule does not exist in the repository.

What do I need to make this libcurl.so available?

Sorry to create an issue even though I expect that I need to change something, not ceema. I don't see another way of asking for support.

Btw: to get to this point, the following installations were necessary on Fedora 37:
sudo dnf install openssl-devel libsodium-devel curlpp-devel mbedtls-devel

[hbruintjes]

Try setting the CMake variable USE_OWN_CURL_LIB to OFF (use ccmake) and reconfigure, that should direct it to use the system version of curl.

[belamu]

Thanks! That helped to remove this error. For all usually-not-(c)make-users like me that end up here:
instead of cmake ../ call cmake -DUSE_OWN_CURL_LIB=OFF ../. ccmake is a interactive way of calling cmake where you can set the variable.

The next error with missing libpurple was fixed with sudo dnf install libpurple-devel.

The next error is a missing zip.h. sudo dnf install zipios-devel zipper-devel does not fix this error but sudo dnf install libzip-devel does fix it.

Then the make runs through (with a lot of -Wpessimizing-move warnings!).

[belamu]

I am not sure if I should close this issue because I looks to me as anyone else should get the same error.

[hbruintjes]

Yeah there is a bit of a usability/documentation issue here, so this can be left open (though the project itself mostly has gone into hiatus).

[belamu]

Since I expect my next problem not to be caused by ceema but my missing knowlegde of pidgin, I'll misuse this issue instead of creating a new one:

I read on the general website that I should "install" a plugin by copying the .so file into ~/.purple/plugins. So I copied the created /build/libceema.so and /build/libceema.so.0.0.1 into this directory (which I had to create).
Then I started pidgin but there is no entry Threema under Accounts and ceema is not shown in the list of plugins (Tools-Plugin) and Threema is not in the list of Protocols in the Add Account dialog.

What do I need to do to use ceema in pidgin?

[belamu]

Yeah there is a bit of a usability/documentation issue here, so this can be left open (though the project itself mostly has gone into hiatus).

Too bad. I was happy to see that there is activity on this project because openMittsu hasn't had activity for 1 year and gave errors for me and so I assumed that maybe the Threema API and the data backup format might have changed.

[hbruintjes]

Plugin probably fails to load. Start pidgin with debug logs (-d) to see if you can spot the issue. That's a pidgin thing though.

And yes, the Threema protocol has been extended, which is also why something happened here. It's a fairly niche product though (or so it seems), so not many alternatives around.

[belamu]

The log indeed shows some information:

(19:19:05) plugins: probing ~/.purple/plugins/libceema.so
(19:19:05) plugins: ~/.purple/plugins/libceema.so is not usable because the 'purple_init_plugin' symbol could not be found.  Does the plugin call the PURPLE_INIT_PLUGIN() macro?

grep gives that purple_init_plugin does exist in your source code:
/src/threepl/prpl/threepl.cpp:extern "C" gboolean purple_init_plugin(PurplePlugin *plugin) {
and purple_init_plugin exists in /build/src/libthreepl.so.

[belamu]

It's a fairly niche product though (or so it seems), so not many alternatives around.

For me it's also just one person that keeps me using Threema, otherwise it would be Signal only. That's quite usable on Desktop. No idea about signal plugin for pidgin.

[hbruintjes]

grep gives that purple_init_plugin does exist in your source code: /src/threepl/prpl/threepl.cpp:extern "C" gboolean purple_init_plugin(PurplePlugin *plugin) { and purple_init_plugin exists in /build/src/libthreepl.so.

Can you check that the symbol was exported with nm? Something along the lines of
nm -D /build/src/libthreepl.so | grep init_plugin

Sometimes something can go wrong during linking. Alternatively, check with ldd that all dependencies can be resolved (e.g., ldd /build/src/libthreepl.so)

[belamu]

Can you check that the symbol was exported with nm? Something along the lines of
nm -D /build/src/libthreepl.so | grep init_plugin
Yes, that exists:

% nm -D build/src/libthreepl.so | grep init_plug
00000000002103b6 T purple_init_plugin

Sometimes something can go wrong during linking. Alternatively, check with ldd that all dependencies can be resolved (e.g., ldd /build/src/libthreepl.so)

% ldd build/src/libthreepl.so
        linux-vdso.so.1 (0x00007ffd727dd000)
        libceema.so.0.0.1 => ~/tmp/ceema-threema-pidgin-plugin-builddir/ceema/build/libceema.so.0.0.1 (0x00007f2a7da00000)
        libzip.so.5 => /lib64/libzip.so.5 (0x00007f2a7e370000)
        libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f2a7dec0000)
        libcurl.so.4 => /lib64/libcurl.so.4 (0x00007f2a7d957000)
        libmbedtls.so.14 => /lib64/libmbedtls.so.14 (0x00007f2a7e33c000)
        libsodium.so.23 => /lib64/libsodium.so.23 (0x00007f2a7de67000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2a7d600000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f2a7d877000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2a7e31a000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f2a7d424000)
        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f2a7e307000)
        liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f2a7d84c000)
        libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f2a7d371000)
        libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f2a7ce00000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f2a7de4d000)
        libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f2a7d2d4000)
        libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007f2a7d2a9000)
        libidn2.so.0 => /lib64/libidn2.so.0 (0x00007f2a7d25c000)
        libssh.so.4 => /lib64/libssh.so.4 (0x00007f2a7cd8e000)
        libpsl.so.5 => /lib64/libpsl.so.5 (0x00007f2a7d838000)
        libssl.so.3 => /lib64/libssl.so.3 (0x00007f2a7cceb000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f2a7cc94000)
        libldap.so.2 => /lib64/libldap.so.2 (0x00007f2a7cc2e000)
        liblber.so.2 => /lib64/liblber.so.2 (0x00007f2a7d24a000)
        libbrotlidec.so.1 => /lib64/libbrotlidec.so.1 (0x00007f2a7e2f4000)
        libmbedx509.so.1 => /lib64/libmbedx509.so.1 (0x00007f2a7cc0b000)
        libmbedcrypto.so.7 => /lib64/libmbedcrypto.so.7 (0x00007f2a7cb85000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2a7e3b4000)
        libunistring.so.2 => /lib64/libunistring.so.2 (0x00007f2a7c9cd000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f2a7c8f0000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f2a7d231000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f2a7d22a000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f2a7c8df000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f2a7c8d8000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f2a7c8c5000)
        libevent-2.1.so.7 => /lib64/libevent-2.1.so.7 (0x00007f2a7c86c000)
        libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f2a7c84c000)
        libbrotlicommon.so.1 => /lib64/libbrotlicommon.so.1 (0x00007f2a7c829000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f2a7c7fc000)
        libcrypt.so.2 => /lib64/libcrypt.so.2 (0x00007f2a7c7c2000)

So it looks as if most went fine. There are two lines without =>: linux-vdso.so.1 and /lib64/ld-linux-x86-64.so.2

I read on the general website that I should "install" a plugin by copying the .so file into ~/.purple/plugins. So I copied the created /build/libceema.so and /build/libceema.so.0.0.1 into this directory (which I had to create).

Did I copy the correct files to the correct place?

[hbruintjes]

Ah, same problem as in #3 I just realized: You need to copy libthreepl.so to the pidgin plugin folder, not libceema. Should fix your issue.

[belamu]

That solves the error shown in the debug log. Now it starts with

(10:00:23) prefs: Reading /home/felix/.purple/prefs.xml
(10:00:23) prefs: Finished reading /home/felix/.purple/prefs.xml
(10:00:23) prefs: purple_prefs_get_path: Unknown pref /pidgin/browsers/command
(10:00:23) dbus: okkk
(10:00:23) plugins: probing /home/felix/.purple/plugins/libthreepl.so
(10:00:24) plugins: probing /usr/lib64/pidgin/timestamp_format.so
...

So "probing" apparently works but it does not show up in the list of Plugins (Menu "Tools"-"Plug Ins") and und "Accounts" there is still only "Manage Accounts", no Threema and when aading a new account, Threema is not listed as a Protocol.
I'll read #3 again more carefully to see if I find anything helpful for me in there. (No, it just worked for @myxor.)

[belamu]

Oh, I'm blind. It did appear in the list of protocols!

[belamu]

The Threema protocol asks for a password. But as far as I am aware, there are no passwords in Threema. What is meant by this password?
I have a data backup and an ID backup. According to your Readme, there should be a menu "Accounts-Threema" but this doesn't show up for me, so that I cannot import my backups. Any idea why? Or is it possible to import the backup somehow via command-line?
As the username I used my Threema-ID.

[belamu]

Tried: username = Threema-ID, password = activation code for apk in threema-shop including "-" in the middle.
Result: accepted but at next start of pidgin a pop-up "Invalid hex input size Invalid username, password or backup string" appears. "Add Buddy" is greyed out.
After opening the "Edit contact" dialog and closing it, the menu "add buddy" is activated. But after adding a buddy with their Threema-ID, pidgin crashes without explanation and without error messages in the log file.

[hbruintjes]

It's not in the readme, but in #6 there is some info on how to log in:

Usually, it is your Threema ID for username, the password for a backup string as password, and then in the account settings specify the backup string. You can fetch the backup string from the app.
Alternatively you can directly use a hex-encoded version of your private key as the password and leave the backup string blank, but you would have to extract it first.

[belamu]

I'm getting there! Yes, this includes all the info I needed to get an "Available" status. When I open Threema on my phone, I also get the info that the account got disabled because I signed it from another location.

When I try to import a data backup (e.g. to get all my contacts) I get an error popup

Unable to import data from backup

Unsupported TLS library used by CURL

I guess that is related to the very beginning of this issue.
As a workaround: is it possible to decrypt the backup externally and import it without decryption and without error?

When I (instead) tried to add a buddy, pidgin closes immediatly after I pressed "Add". But now I have one buddy added successfully. So more testing tomorrow.

Btw: thanks a lot for all your patience guiding me through this on top of your development work. As soon as I've sent and received my first messages, I'll summarise the findings in an PR for the README.

[hbruintjes]

I have no idea how the backup works, I just approved the MR for it :). Could be some crypto is used that the used TLS lib just does not offer.
If adding a buddy fails, try deleting the account in its entirety and reset it.

[belamu]

Then maybe @davidfoerster can help? Instead of reading the entire issue (which includes several unrelated problems):
My current problem is

When I try to import a data backup (e.g. to get all my contacts) I get an error popup
Unable to import data from backup
Unsupported TLS library used by CURL

It might be important that I've called cmake with

instead of cmake ../ call cmake -DUSE_OWN_CURL_LIB=OFF ../

That fixed the error at the very top.

[belamu]

If adding a buddy fails, try deleting the account in its entirety and reset it.

Tried that. Now I get the error when creating the new account "Unable to decrypt HELLO packet". Log says:

(13:36:12) connection: Connection error on 0x55911562e1c0 (reason: 16 description: Unable to decrypt HELLO packet)

I am pretty sure that I've inputed correct password and account backup but am still suspecting that the problem is me in this case.

[davidfoerster]

@belamu I also built with my “own curl lib”:

cmake -DCMAKE_BUILD_TYPE=Release -DUSE_OPENSSL=ON -DUSE_MBEDTLS=OFF -DUSE_OWN_CURL_LIB=OFF ..

I preferred to use the TLS and curl libraries shipped with my distro and ideally wanted ceema to use the same TLS library as curl to avoid issues. Therefore I made sure to install the development files for curl with OpenSSL (libcurl4-openssl-dev).

But I didn’t do any back-up imports since I did that. Before that I used the “official” Ubuntu build and I’m not sure if I ever imported any data from my newly created Threema account. (It seemed easier to simply copy the 4 or so contact IDs by hand.)

[hbruintjes]

If adding a buddy fails, try deleting the account in its entirety and reset it.

Tried that. Now I get the error when creating the new account "Unable to decrypt HELLO packet". Log says:

(13:36:12) connection: Connection error on 0x55911562e1c0 (reason: 16 description: Unable to decrypt HELLO packet)

I am pretty sure that I've inputed correct password and account backup but am still suspecting that the problem is me in this case.

Try these steps, Threema has closed the old gateway: #6 (comment)

[belamu]

Sorry to have left commentless. I tried it again and now followed #6 completely and the HELLO error is gone. For a moment it was errorless but I didn't have to time to add a buddy.
But now I get the error "Remote host closed the connection". From the log:

(16:04:17) dns: Got response for 'g-01.0.threema.ch'
(16:04:17) dnsquery: IP resolved for g-01.0.threema.ch
(16:04:17) proxy: Attempting connection to 185.88.236.90
(16:04:17) proxy: Connecting to g-01.0.threema.ch:5222 with no proxy
(16:04:17) proxy: Connection in progress
(16:04:17) proxy: Connecting to g-01.0.threema.ch:5222.
(16:04:17) proxy: Connected to g-01.0.threema.ch:5222.
(16:04:17) threepl: TRACE - Packet ready, state 1
(16:04:17) threepl: TRACE - Sending session PK:
(16:04:17) threepl: TRACE - Sending nonce:
(16:04:17) threepl: TRACE - onReadyWrite 48
(16:04:17) threepl: TRACE - Written 48 bytes
(16:04:17) threepl: TRACE - Packet handled, state 2, remaining data 0
(16:04:17) threepl: TRACE - onReadyWrite 0
(16:04:17) threepl: TRACE - onReadyRead socket
(16:04:17) threepl: TRACE - Read 80 bytes, expecting 80
(16:04:17) threepl: TRACE - Packet ready, state 2
(16:04:17) threepl: TRACE - Got server nonce:
(16:04:17) threepl: TRACE - Got server session PK:
(16:04:17) threepl: TRACE - Server HELLO OK
(16:04:17) threepl: TRACE - Sending version libceema 0.0.1;;;
(16:04:17) threepl: TRACE - onReadyWrite 144
(16:04:17) threepl: TRACE - Written 144 bytes
(16:04:17) threepl: TRACE - Packet handled, state 3, remaining data 0
(16:04:17) threepl: TRACE - onReadyRead socket
(16:04:17) connection: Connection error on 0x562c7f5da840 (reason: 0 description: Remote host closed the connection)
(16:04:17) account: Disconnecting account (0x562c7e8a5230)
(16:04:17) connection: Disconnecting connection 0x562c7f5da840
(16:04:17) connection: Destroying connection 0x562c7f5da840

Also my Threema Libre on my phone doesn't say anything about deactivation of the account.

My settings:

Username: my Threema-ID
Password: the password of the ID-Backup
Backup string: long ID-Backup with -
Server: g-01.0.threema.ch
Port: 5222
uncheck Use application gateway [I forgot that before]

Unless you have another idea I assume that Threema changed enough to not support other clients.
Thank you for all your help.

[davidfoerster]

@belamu You may want to redact those logs to mask or omit your Threema ID. (Editing your comment will leave previous versions accessible to the public. Better to remove your comment and write another.)

[belamu]

Thanks @davidfoerster . I thought I've checked but obviously missed that.