precompiled: HEIC support not working?

[feld]

I'm seeing this on MacOS and Linux with 0.31.1

iex(3)> Vix.Vips.Image.supported_saver_suffixes()
{:ok,
 [".avif", ".heif", ".heic", ".tiff", ".tif", ".webp", ".jfif", ".jpe", ".jpeg",
  ".jpg", ".png", ".szi", ".dz", ".gif", ".vips", ".v", ".raw", ".mat", ".csv"]}

ok, look like support should be there.

iex(4)> {:ok, image} = Vix.Vips.Image.new_from_file("test/fixtures/image.heic")
iex(5)> Vix.Vips.Image.write_to_stream(image, ".jpg") |> Stream.into(File.stream!("./out.jpeg")) |> Stream.run
** (Vix.Vips.Image.Error) Failed to write to target
    (vix 0.31.1) lib/vix/vips/image.ex:583: anonymous fn/1 in Vix.Vips.Image.write_to_stream/2
    (elixir 1.14.5) lib/stream.ex:1625: Stream.do_resource/5
    (elixir 1.14.5) lib/stream.ex:584: Stream.do_into/4
    (elixir 1.14.5) lib/stream.ex:689: Stream.run/1
    iex:5: (file)

Fails?

On FreeBSD when I am forced to compile against the system provided vips library:

iex(4)> {:ok, image} = Vix.Vips.Image.new_from_file("test/fixtures/image.heic")
{:ok, %Vix.Vips.Image{ref: #Reference<0.1910391208.3904503840.259935>}}
iex(5)> Vix.Vips.Image.write_to_stream(image, ".jpg") |> Stream.into(File.stream!("./out.jpeg")) |> Stream.run
:ok

> file out.jpeg
out.jpeg: JPEG image data, Exif standard: [TIFF image data, big-endian, direntries=5, orientation=upper-left, xresolution=74, yresolution=82, resolutionunit=2], baseline, precision 8, 700x476, components 3

So, it worked here on FreeBSD.

Annoyingly on MacOS (homebrew) and Debian, libvips does not come linked against libheif so that adds a big hurdle to working around this 😭

[akash-akya]

@feld is it possible to share the image? Prebuilt binaries does support AVIF images in HEIF container I think. If the image is of HEVC compression then it wont work.

Main reason why it is not bundled with the prebuilt binaries (and with platform packages) is that patent licensing around HEVC/x265 seems to be messy.

You can check the compression by

{:ok, img} = Vix.Vips.Image.new_from_file(path)
Vix.Vips.Image.header_value(img, "heif-compression")

[feld]

Yeah, it's just a basic test file we use

edit: if github mangled this, here's a link from our repo

https://git.pleroma.social/pleroma/pleroma/-/blob/feat/vips-heif-upload-filter/test/fixtures/image.heic?ref_type=heads

[feld]

iex(1)> {:ok, img} = Vix.Vips.Image.new_from_file("test/fixtures/image.heic")
{:ok, %Vix.Vips.Image{ref: #Reference<0.267671389.4215406613.43045>}}
iex(2)> Vix.Vips.Image.header_value(img, "heif-compression")
{:ok, "hevc"}

you're right about it being HEVC... now the question is, what does Apple normally use? Time to find a sample

[feld]

The files I'm finding from iMessages from friends which are HEIC format also report as HEVC...

I see in the FreeBSD project we're shipping libheif with HEVC enabled. In 2020 we stopped restricting things like LAME encoder with this note:

Patents are a complicated topic, and their regulation varies depending on
jurisdiction. Patents  are not necessarily related to the license and so
should not be connected to the license framework.

As a project we will officially remove all patent limitations within the
ports tree and leave it to the user or consumer to deal with their local
legislation to determine if they can use the software without legal
restrictions.

Approved by:	core

I don't know what to say other than I hope you reconsider and can ship full HEIC/HEIF support as it would make working with this format a lot easier.

[feld]

I was just thinking about this again and I think a reasonable solution may be to make it possible to self-host the Pre-compiled NIF, perhaps by setting an ENV that is obeyed at build time?

[akash-akya]

Perhaps I can make a nix flake with dependencies. User can use that to install libvips and then configure vix to use platform provided libvips. Then again, I might not be actively maintaining it, it will be like a path to install libvips. Ideally the nix flake should work with both Macos & Linux. Does that help?

[akash-akya]

possible to self-host the Pre-compiled NIF

That would be hard. I am using cc_precompiler to handle NIF pre-compilation and it heavily depends on the current package. We basically have to rewrite all that logic to support this type of usecase.

[venkatd]

@akash-akya thanks for this incredible library.

We need hevc support as well. It's a shame about all those licensing restrictions.

If you have are put together a nix flake that makes it easy to install a build with hevc support, I'd be happy to pay for the time it takes you to get that up if that is something interesting to you.

[akash-akya]

Hey @venkatd, Thanks for using the library.
I'll spend some time and share the nix flake by this weekend :)

[venkatd]

Great! I'll send you an email

[akash-akya]

Hey @venkatd, sorry for the delayed response.

NixOS repo already provides libvips package with x265/HEIC support. So we can just use it. Here's a step-by-step guide to use it:

1. Install and set up Nix with Flakes support.

2. Create a flake.nix file with the following content:

{
  description = "Using Nix libvips with Elixir";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
      in {
        devShells.default = pkgs.mkShell {
          buildInputs = [
            pkgs.vips
            pkgs.elixir # optional. I am using the Elixir provided by Nix
            pkgs.pkg-config
          ];
        };

        formatter = pkgs.nixfmt-rfc-style;
      }
    );
}

3. Create a sample Elixir script thumb.exs:

Mix.install([{:vix, "~> 0.23", force: true}],
  system_env: [
    # must be set to PLATFORM_PROVIDED_LIBVIPS, otherwise Vix uses prebuilt libvips
    {"VIX_COMPILATION_MODE", "PLATFORM_PROVIDED_LIBVIPS"}
  ]
)

alias Vix.Vips.{Image, Operation}

{:ok, thumb} = Operation.thumbnail("image.heic", 300)
:ok = Image.write_to_file(thumb, "thumbnail.jpg", Q: 90, strip: true, interlace: true)

4. Open Nix develop shell:

$ nix develop 

This will install required Nix dependencies and create a flake.lock file.

5. Execute the script inside the Nix develop shell:

$ elixir thumb.exs input.heic

Note: You may need to modify the flake.nix for production deployment.

---

If you want greater control over libvips and its dependencies, you can create a new flake and use that instead of using the default libvips. The custom libvips pkg might look like this

{
  description = "Flake for building libvips";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
    libvips-src = {
      url = "github:libvips/libvips/v8.16.0";
      flake = false;
    };
  };

  outputs =
    {
      self,
      nixpkgs,
      flake-utils,
      libvips-src,
    }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = import nixpkgs { inherit system; };

        libvips = pkgs.stdenv.mkDerivation {
          pname = "libvips";
          version = "8.16.0";

          src = libvips-src;

          nativeBuildInputs = with pkgs; [
            gobject-introspection
            meson
            ninja
            pkg-config
          ];

          buildInputs =
            with pkgs;
            [
              glib
              libxml2
              expat

              # Optional dependencies
              cfitsio
              cgif
              fftw
              imagemagick
              giflib
              lcms2
              libarchive
              libde265
              libexif
              libgsf
              libheif
              libimagequant
              libjpeg
              libjxl
              libpng
              librsvg
              libspng
              libtiff
              libtiff
              libwebp
              matio
              openexr
              openjpeg
              openslide
              pango
              poppler
            ]
            ++ lib.optionals stdenv.hostPlatform.isDarwin [
              ApplicationServices
              Foundation
            ];

          # Required by .pc file
          propagatedBuildInputs = with pkgs; [
            glib
          ];

          mesonFlags = with pkgs.lib; [
            (mesonEnable "pdfium" false)
            (mesonEnable "nifti" false)
            (mesonBool "deprecated" false)
            (mesonBool "gtk_doc" false)
          ];

          meta = with pkgs.lib; {
            homepage = "https://www.libvips.org/";
            description = "Image processing system";
            license = licenses.lgpl2Plus;
            maintainers = with maintainers; [
              akash-akya
            ];
            pkgConfigModules = [
              "vips"
              "vips-cpp"
            ];
            platforms = platforms.unix;
            mainProgram = "vips";
          };
        };
      in
      {
        packages.default = libvips;

        # Expose as an overlay
        overlays.default = final: prev: {
          libvips = libvips;
        };

        # Development shell
        devShells.default = pkgs.mkShell {
          buildInputs = [
            self.packages.${system}.default
            pkgs.pkg-config
          ];
        };

        formatter = pkgs.nixfmt-rfc-style;
      }
    );
}

I can assist you with further details if you want.