nginx serves corrupted files when files edited in host mounted volume #68

stefanfoulis · 2014-08-22T11:21:09Z

I get weird file corruption when mounting /var/www into this container. It seems to be specific to nginx in combination with docker-osx (not sure if the same happens with other virtualisation countainers that mount stuff from the host machine into the VM). I could not reproduce this on a pure linux/docker host. I had this on MacOSX 10.9.4 and a fresh docker-osx install.

$ docker-osx shell
$ mkdir ~/Sites/nginxtest
$ docker run --rm -i -t -v /Users/stefanfoulis/Sites/nginxtest:/usr/local/nginx/html -p 0.0.0.0:42002:80 nginx:latest
$ curl http://172.16.42.43:42002/index.html
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.7.1</center>
</body>
</html>

All fine so far.

$ echo "Hi there" > ~/Sites/nginxtest/index.html
$ curl http://172.16.42.43:42002/index.html
Hi there

So far so good. Let's add something to the file

$ echo "More stuff" >> ~/Sites/nginxtest/index.html
$ curl http://172.16.42.43:42002/
Hi there

The new line is missing. Even stopping the container and starting a new one will not have the change show up.

^C%
$ docker run --rm -i -t -v /Users/stefanfoulis/Sites/nginxtest:/usr/local/nginx/html -p 0.0.0.0:42002:80 nginx:latest
$ curl http://172.16.42.43:42002/
Hi there

And what is super weird, I can connect to the container with bash, run nginx in the backround and observe the files changing in the filesystem. But nginx does not pick them up.

$ docker run --rm -i -t -v /Users/stefanfoulis/Sites/nginxtest:/usr/local/nginx/html -p 0.0.0.0:42002:80 nginx:latest /bin/bash
root@e9e8fae26a58:/usr/local/nginx/html# nginx &
[1] 8
root@e9e8fae26a58:/usr/local/nginx/html# cat index.html
Hi there
More stuff

but at the same time from the host:

$ curl http://172.16.42.43:42002/index.html
Hi there

Deleting the file gets picked up though

$ rm ~/Sites/nginxtest/index.html
$ curl http://172.16.42.43:42002/index.html
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.7.1</center>
</body>
</html>

And now it gets interesting. Overwriting a file with a shorter content will show the content of the old file, but truncated at the length of the new file.

$ echo "And now for something completely different." > ~/Sites/nginxtest/index.html
$ curl http://172.16.42.43:42002/index.html
And now for something completely different.
$ echo "Now a shorter text" > ~/Sites/nginxtest/index.html
$ curl http://172.16.42.43:42002/index.html
And now for somethi%
$ echo "A way longer text that contains completely different content. Did I mention it is longer?" > ~/Sites/nginxtest/index.html
$ curl http://172.16.42.43:42002/index.html                                                                                        
And now for something completely different.

It looks like it's tied to the first request done with a new file. If you delete the file and make at least one request for that file and get a 404 and then re-add it, the new file contents will be picked up.

It seems to be a combination of docker-osx and nginx (I experienced this with the orchardup/nginx docker image as well. We did a quick test with an apache image, which did not show this behaviour.

My wild speculation: I suspect nginx is doing some deep magic in the way it detects file changes and caches it somehow (but apparently not just in memory, otherwise it would not survive container destruction and restart). Something with the docker abstraction of the volumes (and maybe just in combination of using a volume that was shared to the vm from the host os) and how nginx reads the files.

I'm linking this issue from docker and the official nginx docker image as well, as I'm not sure where the issue originates.

Some additional environment information:

docker version
Client version: 1.1.1
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): bd609d2
Server version: 1.1.1
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): bd609d2

The text was updated successfully, but these errors were encountered:

ghost · 2014-08-27T14:06:29Z

I am having the exact same issue. Same versions.. same behaviour.

I think it is related to the VirtualBox sendfile issue.

What solved it for me was adding sendfile off; to the location tag in my site configuration.
Which is strange because I had it already disabled in my main nginx config (which used to work in the past).

amosyuen · 2015-10-26T00:11:36Z

same problem, mrmaxon's workaround works for me

joserobleda · 2015-11-23T08:29:34Z

same problem here, and it works with sendfile off;

cor · 2016-09-12T14:35:20Z

I'm having the same issue, this really needs to be fixed

virtustilus · 2017-07-20T05:37:26Z

I had to restart docker-machine and after that all became ok:
docker-machine stop default

This was referenced Aug 22, 2014

nginx serves corrupted files when files edited in host mounted volume moby/moby#7682

Closed

nginx serves corrupted files when files edited in host mounted volume docker-library/nginx#8

Closed

stefanfoulis changed the title ~~file changes not being picked up with mounted volume from host and nginx~~ nginx serves corrupted files when filed edited in host mounted volume Aug 22, 2014

stefanfoulis changed the title ~~nginx serves corrupted files when filed edited in host mounted volume~~ nginx serves corrupted files when files edited in host mounted volume Aug 22, 2014

shirk3y mentioned this issue Mar 19, 2015

Container with host mounted volume serving corrupt files boot2docker/osx-installer#100

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nginx serves corrupted files when files edited in host mounted volume #68

nginx serves corrupted files when files edited in host mounted volume #68

stefanfoulis commented Aug 22, 2014

ghost commented Aug 27, 2014

amosyuen commented Oct 26, 2015

joserobleda commented Nov 23, 2015

cor commented Sep 12, 2016

virtustilus commented Jul 20, 2017

nginx serves corrupted files when files edited in host mounted volume #68

nginx serves corrupted files when files edited in host mounted volume #68

Comments

stefanfoulis commented Aug 22, 2014

ghost commented Aug 27, 2014

amosyuen commented Oct 26, 2015

joserobleda commented Nov 23, 2015

cor commented Sep 12, 2016

virtustilus commented Jul 20, 2017