Package building assistance

[ehawman]

Howdy! I want to help create packages myself but I'm struggling with the instructions. I'm not familiar enough with the process to understand why certain things are being done, so I'm having trouble translating them to a different environment. Please bear with me as my questions will likely look like someone fumbling in the dark.

I'm using a Mac machine, but ultimately I think it would be beneficial to have the instructions cover all OSes. I'd be happy to walk through this process on multiple machines and really iron out the instructions so anyone can follow.

The first package I'd like to create is for micro so let's use that as an example. Currently I can install it on Unraid with:

curl https://getmic.ro | bash  # Pipes the install script to bash, installing to ./micro.
mv ./micro /usr/bin/micro  # Moves ./micro to /usr/bin

For this operation, I'm assuming I should download the source code from their releases page, which I've done and I have unzipped in ~/example/micro-2.0.10. Or am I taking a built/installed executable and zipping that into a package? Should I be using the /usr/bin/micro from my Unraid server directly?

1. copy tar-1.13 from unRAID to /bin/tar-1.13 on laptop

   Is this used by makepkg? Is it referencing tar-1.13 or just tar? Will any version do or does it have to be 1.13?

   Both Mac and Windows support tar natively, and I believe this is true of 99% of Linux distros as well. Can this step be eliminated entirely?

2. copy makepkg from unRAID to /sbin/makepkg on laptop

   Mac locks down /sbin/ quite thoroughly. You can get to it but it requires rebooting into a special safe mode. I put makepkg in /usr/local/bin/ and it appears on my PATH. Should that be enough?

3. I modified gfjardim's pkg_build.sh to take an argument and add arch type plus modified the lettering function

   Is this strictly for reference or should I be doing something with pkg_build.sh at this point? Presumably pkg_build.sh should be somewhere specific, but where? The only one in this repo is /source/SlackBuild/rmlint/pkg_build.sh. Is that right? Should I invoke it somehow?

4. To compile the packages for unRAID plugin run from the source directory

   sudo ./mkpkg plugin

   This creates a slackware compliant package and md5 in the archive directory with the date as the version number

   And adds the date to the plugin file in plugins directory.

   e.g. plugin-2017.06.09-x86_64-1.txz and plugin-2016.06.09-x86_64-1.md5

   Oh boy here we go. What is the source directory in this context? Is it ~/example/micro-2.0.10? sudo ./mkpkg plugin predictably says that ./mkpkg can't be found. For grins I tried it with sudo makepkg micro and it said ERROR: Package extension .micro is not supported. I put my executable in /source/, ran sudo ./mkpkg micro and it dumped an empty .txz in /archive/ along with an md5.

At this point I'm pretty stumped. I'm sure the instructions are complete if you have enough familiarity with this kind of thing, but as you can tell I'm clearly missing the important stuff. Please help :)

[dmacias72]

Those instructions are for creating a slackware package for a plugin. Putting any binaries and webui pages from the SRC directory into a compliant installable slackware package with a date as the version number and modifies the plugin plg file with the package name.

[dmacias72]

The binary packages I personally compile for unraid I do so in a VM with the latest version of unraid. I install my DevPack plugin which contains the headers that they remove from unraid for the binaries that are built in. DevPack also includes many of packages necessary for compiling.

So first you need a VM or a install of unraid. I don't recommend your main install of unraid. But you could use it and the reboot once you're done. You need to create a build directory on you flash drive of the VM or unraid you're using to compile. Next install DevPack. Then you need to go to slackbuilds.org and search for the package you want to compile. In this case micro. I only saw a slackbuild for slackware 15 but it should work. You need to download the micro.tar.gz slackbuild file and extract it into the build directory you created on your flash drive in its own directory micro. (not sure if the slackbuild file has paths built in) It contains the micro.Slackbuild file which is the script to compile micro. I edit the Slackbuild file to output txz instead of tgz. Next you can download the source from slackbuild or the linux 64 source from github. You need the tar.gz format. Place that in the build/micro. Next you ssh into the vm and go to build/micro. The run sh micro.Slackbuild and see what errors or headers it complains about. Maybe you'll get lucky and it will compile. It will create the package in /tmp. You need to move that to your flash drive or somewhere other than temporary memory.

[ehawman]

@dmacias72 thank you very much for the writeup. That clears a ton of things up.

My understanding of Unraid licensing is that starting the array requires a paid license, while booting into the OS shouldn't require anything. I'm unsure if I can install DevPack to an unlicensed Unraid install, but I'll look into it tonight. Perhaps I can install it on a trial then continue using it afterwards?

I've done a quick writeup here which documents my understanding of what you've written. Please correct me where I've gone astray. I'm missing pkg_build.sh and sudo ./mkpkg plugin in here as I'm not 100% sure where they come in.

Environment Setup:

1. Setup an Unraid VM
   • I can fill in more info on this later. Do you think running a VM inside Unraid would cause problems? I know console access is baked in nicely, and obviously anyone following this tutorial would have access to an Unraid instance.
2. Install DevPack
   • I'm assuming that DevPack includes package creation tools. As such, no NerdPack needed, right?
3. Create a build directory
   • /boot/package_builds/

Build Package:

1. Find desired package on slackbuilds.org, download the "SlackBuild", and extract it into the above dir.
   • https://slackbuilds.org/slackbuilds/15.0/development/micro.tar.gz
   • 📂package_builds
     ┗ 📂micro
     ┃ ┣ 📜doinst.sh
     ┃ ┣ 📜micro.info
     ┃ ┣ 📜micro.SlackBuild
     ┃ ┣ 📜README
     ┃ ┗ 📜slack-desc
2. In {package_name}.SlackBuild, change PKGTYPE=${PKGTYPE:-tgz} to `PKGTYPE=${PKGTYPE:-txz}
3. Download the x64 Source and extract the contents to the build folder
   • https://github.com/zyedidia/micro/releases/download/v2.0.10/micro-2.0.10-linux64.tar.gz
   • 📂package_builds
     ┗ 📂micro
     ┃ ┣ 📂micro-2.0.10
     ┃ ┃ ┣ 📜LICENSE
     ┃ ┃ ┣ 📜LICENSE-THIRD-PARTY
     ┃ ┃ ┣ 📜micro
     ┃ ┃ ┣ 📜micro.1
     ┃ ┃ ┣ 📜micro.desktop
     ┃ ┃ ┣ 📜micro.svg
     ┃ ┃ ┗ 📜README.md
     ┃ ┣ 📜doinst.sh
     ┃ ┣ 📜micro.info
     ┃ ┣ 📜micro.SlackBuild
     ┃ ┣ 📜README
     ┃ ┗ 📜slack-desc
4. Access VM console and cd into it
   • cd /boot/package_builds/micro
5. Run the SlackBuild file. If it fails, attempt to navigate errors and reattempt until successful.
   • ./micro.SlackBuild
6. Copy resulting package back to build folder
   • cp -r /tmp/????.txz /boot/package_builds/micro/result/????.tzx

Prepare Git Pull Request:

1. Either fork https://github.com/dmacias72/unRAID-NerdPack, or pull/fast-forward your existing fork.
2. Create a branch for the package you want to create
   • micro-2.0.10
3. Checkout the branch to your local machine (not the VM)
4. Move ????.txz to /packages/[version-of-your-Unraid-VM]/
   • scp root@yourunraidserver:/domains/unraid_build_vm_example_volume/boot/package_builds/micro/result/????.tzx /path/to/git/folder/unRAID-NerdPack/packages/6.10/????.tzx
5. Create pull request. (Do you care about Conventional Commits?)

Again, thank you for taking the time out to assist me with this. Hopefully I can pay it forward.

[dmacias72]

Looks like you got it. The pkg_build.sh and mkpkg were only for creating packages for plugins. When I make a change to the NerdPack or other plugin I use them to create the new plugin package. I also forgot, you need to edit the micro.Slackbuild file and make changes at the top of the file to point to the new version of source if it's different. You don't need to extract the micro source either. Just place the micro-2.0.10-linux64.tar.gz in the same directory as micro.Slackbuild.

While I have a couple unraid licenses, I never went through the process of setting up the VM using a second USB stick because at the time have 2 unraid usb sticks when booting wouldn't work constantly. Maybe the option to select the main boot USB wasn't complete yet. Now I think you can pick a particular USB to boot from and it will work. So I boot the VM with a virtual image. Can't remember if I just cloned my main usb or not. You are correct you can't start the array and unraid will complain about the invalid license. But you can install plugins and compile all you want. I just make copies of the virtual drive and upgrade so I can have multiple copies and run different versions.

You can just do a pull request on the main branch. Just place the package in the correct unraid version directory. Also you can add the source slackbuild files under src like I've done for some packages.

Also thanks very much for trying to do this. I just don't have much time anymore and rarely get on the forums.

[ehawman]

@dmacias72 Sweet. Ok Take II

---

Environment Setup:

1. Setup an Unraid VM
   • Create a copy of your drive and install it as a VM onto your Unraid server. # TODO: Make full writeup of this step
   • If you have already created a drive image for a desired Unraid version, use it here. (You'd have multiples. Please read below)
2. Setup SSH # TODO: Make a full writeup of this step
3. Install DevPack
4. Create a build directory
   • /boot/package_builds/

Build Package:

1. Download the x64 Source to your build dir.
   • https://github.com/zyedidia/micro/releases/download/v2.0.10/micro-2.0.10-linux64.tar.gz
   • 📂package_builds
     ┗ 📂micro
     ┃ ┗ 📦micro-2.0.10-linux64.tar.gz
2. Find desired package on slackbuilds.org, download the "SlackBuild", and extract it into the your build dir.
   • https://slackbuilds.org/slackbuilds/15.0/development/micro.tar.gz
   • 📂package_builds
     ┗ 📂micro
     ┃ ┣ 📦micro-2.0.10-linux64.tar.gz
     ┃ ┣ 📜doinst.sh
     ┃ ┣ 📜micro.info
     ┃ ┣ 📜micro.SlackBuild
     ┃ ┣ 📜README
     ┃ ┗ 📜slack-desc
3. In {package_name}.SlackBuild, change PKGTYPE=${PKGTYPE:-tgz} to PKGTYPE=${PKGTYPE:-txz}
4. If the version of the source code is different than the version listed in the SlackBuild file, update the SlackBuild to match.
   • VERSION=${VERSION:-2.0.10}
5. Access VM console via SSH or webterminal and cd into your build folder
   • cd /boot/package_builds/micro
6. Run the SlackBuild file. If it fails, attempt to navigate the errors, rerun, and repeat until successful.
   • ./micro.SlackBuild
7. Copy resulting package back to build folder
   • cp -r /tmp/micro-2.0.10.txz /boot/package_builds/micro-2.0.10.tzx

Prepare Git Pull Request:

1. Either fork https://github.com/dmacias72/unRAID-NerdPack, or pull/fast-forward your existing fork's Main branch.
2. Create a branch for the package you want to create
   • micro-2.10.0
3. Checkout the branch to your local machine (not the VM)
4. Copy yourpackage.txz from the Unraid VM to the fork on your local machine, in /packages/[version-of-your-Unraid-VM]/
   • scp root@yourunraidserver:/domains/unraid_build_vm_example_volume/boot/package_builds/micro-2.0.10.tzx /path/to/git/folder/unRAID-NerdPack/packages/6.10/micro-2.0.10.tzx
5. Copy the Source folder from the Unraid VM to the fork on your local machine, in `/source/SlackBuild/{your package name}/
   • scp -r root@yourunraidserver:/domains/unraid_build_vm_example_volume/boot/package_builds/micro/ /path/to/git/folder/unRAID-NerdPack/source/SlackBuild/micro/
6. To create packages for multiple Unraid versions, save a copy of the VM drive as the Unraid version number for future use, then go all the way back to Step 1 and restart with a different Unraid version as your VM. Eventually you'll have a nice stable of Unraid images to build packages for, and this process won't suck nearly as much.
7. Create pull request. Be sure to specify which Unraid versions you've built packages for.
8. Once your pull request has been accepted, feel free to delete your branch (on both origin and local).

---

I left the creation of a new branch branch management in there since A) It's best practice and B) it can really help compartmentalize things should someone be working on multiple different packages at a time (which people will definitely be doing since it makes sense to build 5 packages on the particular version of Unraid currently installed before moving to the next one vs cycling through versions X times per package, 5 different times)

Unfortunately I ran out of time to test the VM (and the whole process) but I should be able to carve some time out tomorrow. Believe me, I know the horrific burden of discovering I've signed up for a forever task. Hopefully these instructions will allow absolutely anyone to build packages, meaning your job will just be to glance over/approve the pull requests and do periodic package updates.

If you care to peel back the curtain a bit more on what happens when my instructions end, perhaps there's some more automated releasing that can be done. I have less than zero knowledge about how releasing plugins to Community Apps works, but walk me through it and maybe we can completely automate the process.

Heck, how sweet would it be to feed a script the URL to download the source tar.gz, the SlackBuild tar.gz, and then sit back and let it crank out 15 different packages for 15 different Unraid versions, and finally push the entire shebang to the Community Apps store?

[dmacias72]

It sounds great. For Community apps, I believe you have to create a repo with xml files pointing to the plugin repos. Community apps polls the repo for changes. Nerdpack just polls the packages repo for changes. So if you update a package, once it's pages to github, it will show up in unraid's webui. When you mentioned before about branches it got me thinking that all the packages should be separated out into their own repo for unraid packages. Then have different branches for different version of unraid. Nerdpack repo should just be for code changes. I really never add packages to older versions. Most people are either on the latest or version just before or testing the next version.

[ehawman]

Ok well I had a beautiful dream but I keep stepping on a rake at Stage 1.

I cannot, for the life of me, get an Unraid VM working on a Windows or Mac machine. And that's using every human-flexible trick at my disposal, not even trying to script the process.

How did you build your VM?

[dmacias72]

I run my Unraid VMs on unraid. I followed some guide on the forum a long time ago but here's a link to a guide I just found. https://docs.ibracorp.io/virtual-unraid/

Edit:
You don't necessarily have to run all this in a VM. But you run the risk of data loss if you install the wrong package and the server locks up or you manually have to shut down. If you did run it on a regular server, I would just recommend disabling all the devpack packages and restarting when done.

[GHTGXL]

After try a few hours later.

I mostly know what to do,but the package I make still need redo.

OK, The first one is I using ubuntu, so maybe some can use?

Second, I try use dev tools in unraid, but I try to make borg package in unraid, missing openssl. So cant make package. Even I check openssl version, and cli have show the version.

So I swap to ubuntu, See Readme say copy makepkg to /sbin/makepkg. I find makepkg also in /sbin/makepkg in unraid, Copy this to /sbin/makepkg in ubuntu, Also change Chown(owner and group) to root in ubuntu. and change chmod in mc.

on: read by owner, write by owner, execute/search by owner, read by group, execute/searchby group, read by other, execute/searchby other.

I follow SlackBuild>download the build I want>follow #66 (comment) >Run xxx.slackbuild> get xxx.txz

But I only get xxx.txz, not have md5, and the package still need redo. Because it still don't working. And size is strange, from nerdpack is 4.2MB, from me is less for 1MB.

Who Know how to fix this?

[VergilGao]

After try a few hours later.

I mostly know what to do,but the package I make still need redo.

OK, The first one is I using ubuntu, so maybe some can use?

Second, I try use dev tools in unraid, but I try to make borg package in unraid, missing openssl. So cant make package. Even I check openssl version, and cli have show the version.

So I swap to ubuntu, See Readme say copy makepkg to /sbin/makepkg. I find makepkg also in /sbin/makepkg in unraid, Copy this to /sbin/makepkg in ubuntu, Also change Chown(owner and group) to root in ubuntu. and change chmod in mc.

on: read by owner, write by owner, execute/search by owner, read by group, execute/searchby group, read by other, execute/searchby other.

I follow SlackBuild>download the build I want>follow #66 (comment) >Run xxx.slackbuild> get xxx.txz

But I only get xxx.txz, not have md5, and the package still need redo. Because it still don't working. And size is strange, from nerdpack is 4.2MB, from me is less for 1MB.

Who Know how to fix this?

here is my script named libsbc.SlackBuild to build libsbc for unraid

#!/bin/bash

PRGNAM=libsbc
SRCNAM=libsbc
VERSION=1.5.1
BUILD=${BUILD:-1}
TAG=${TAG:-_SBo}
COMMIT=909a9bdf7ab143e1f0baaf9736baebd3cd79aacf
SOURCE=https://git.kernel.org/pub/scm/bluetooth/sbc.git/snapshot/sbc-$COMMIT.tar.gz

CWD=$(pwd)
OUTPUT=${OUTPUT:-$CWD/out}
PKG=${PKG:-$OUTPUT/$SRCNAM}
DESTDIR=${DESTDIR:-$PKG/dest}

if [ -z "$ARCH" ]; then
  case "$( uname -m )" in
    i?86) ARCH=i586 ;;
    arm*) ARCH=arm ;;
       *) ARCH=$( uname -m ) ;;
  esac
fi

if [ "$ARCH" = "i486" ]; then
  SLKCFLAGS="-O2 -march=i486 -mtune=i686"
  LIBDIRSUFFIX=""
elif [ "$ARCH" = "i686" ]; then
  SLKCFLAGS="-O2 -march=i686 -mtune=i686"
  LIBDIRSUFFIX=""
elif [ "$ARCH" = "x86_64" ]; then
  SLKCFLAGS="-O2 -fPIC"
  LIBDIRSUFFIX="64"
else
  echo -e "not support arch $ARCH"
  exit 1
fi

set -e

mkdir -p $OUTPUT $PKG
rm -rf $PKG/*
wget -O- $SOURCE | tar xz -C $PKG --strip-components 1
cd $PKG
chown -R root:root .
find -L . \
  \( -perm 777 -o -perm 775 -o -perm 750 -o -perm 711 -o -perm 555 \
  -o -perm 511 \) -exec chmod 755 {} \; -o \
  \( -perm 666 -o -perm 664 -o -perm 640 -o -perm 600 -o -perm 444 \
  -o -perm 440 -o -perm 400 \) -exec chmod 644 {} \;

autoreconf --install
mkdir -p build && cd build

CFLAGS="$SLKCFLAGS" \
CXXFLAGS="$SLKCFLAGS" \
  ../configure \
  --prefix=/usr \
  --libdir=/usr/lib${LIBDIRSUFFIX} \
  --sysconfdir=/etc \
  --build=$ARCH-slackware-linux
make -j$(nproc) && make install DESTDIR=$DESTDIR

cd ..

mkdir -p $DESTDIR/usr/doc/$PRGNAM-$VERSION
cp -a COPYING COPYING.LIB README $DESTDIR/usr/doc/$PRGNAM-$VERSION
cat $CWD/$PRGNAM.SlackBuild > $DESTDIR/usr/doc/$PRGNAM-$VERSION/$PRGNAM.SlackBuild

cd $DESTDIR
/sbin/makepkg -l y -c n $OUTPUT/$PRGNAM-$VERSION-$ARCH-$BUILD$TAG.${PKGTYPE:-txz}

Everything works fine except that the md5 is not generated