Volume storage not available while building from Docker image? | Railway | Page 1

grand sage Oct 21, 2023, 6:27 PM

#

I'm looking to deploy the mediawiki Docker image as a service. After deploying, it correctly starts up apache as a web service, and the first connection is then walked through a MediaWiki set-up process that asks for relevant info like DB authentication and customizable options for the website. At the end of the process, a file LocalSettings.php is generated that must be manually downloaded and then re-uploaded to the root folder for the website (i.e., /var/www/html).
Since this calls for persistent information to be available between re-deploys of the service, it calls for a volume. But whenever I attach a volume, it appears to clobber the contents of /var/www/html that already exist in the container: after mounting a new volume to my service in Railway, attempting to connect to the website gives me a 'Forbidden' access error since it cannot find index.php. This also happens when I mount the volume to an empty service before loading the MediaWiki docker image.

Is there a proper way to mount a volume that would let me persist files generated by the container's build process alongside a new file like LocalSettings.php that I could add later (perhaps through the FileBrowser template)?

vapid coralBOT Oct 21, 2023, 6:27 PM

#

Project ID: N/A

gleaming pecanBOT Oct 21, 2023, 6:27 PM

#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

grand sage Oct 21, 2023, 6:27 PM

#

N/A

smoky creek Oct 21, 2023, 8:03 PM

#

the volume is not available during build, but I don't think that's the issue here

#

you are deploying a docker image, there really isn't a build stage since the image is already built

grand sage Oct 21, 2023, 8:56 PM

#

Fair point about the build part not being the issue. But the underlying issue remains, that I can't seem to attach a volume without it changing/removing what's already there.
I can deploy a container locally doing something like
docker run -d ... --mount source=myvolume,target=/var/www/html mediawiki:latest
and this works as I would have expected.

#

But I'm not sure how to achieve the same effect on Railway.

smoky creek Oct 21, 2023, 9:22 PM

#

mediawiki needs two volumes, and wants a volume mounted to /var/www/html/LocalSettings.php never a good idea in the first place, so i dont think you will be able to run this on railway

grand sage Oct 21, 2023, 9:25 PM

#

I’ll take you at your word that it might not be possible on Railway atm.
But can you elaborate on what you mean by mediawiki needing two volumes?

smoky creek Oct 21, 2023, 9:30 PM

#

my bad i thought you would have seen the dockercompose file
https://hub.docker.com/_/mediawiki

grand sage Oct 22, 2023, 5:14 AM

#

I saw that docker compose file, but isn't that just an example compose file? The paragraph above specifies that the dockerhub image doesn't include any volumes.

#

And since both the images directory and the LocalSettings.php file itself are contained within /var/www/html I would have thought it sufficient for deploying on Railway (if not exactly the intention for the dockerhub image) that I optionally include a single volume mounted at /var/www/html, encapsulating both the images dir and LocalSettings file.

smoky creek Oct 22, 2023, 5:35 AM

#

guess mediawiki is setup in a way that mounting to that breaks

grand sage Oct 22, 2023, 6:17 AM

#

😦

#

This only happens on Railway though. As I mentioned, using the same image from dockerhub and running on my own PC docker run... with the same mount point gives me a working container that behaves as expected: all of the usual stuff that MediaWiki puts in /var/www/html is present and accounted for, except it's all sitting on the mounted volume. At that point I can add the LocalSettings file and know it will persist. I don't know why this isn't possible on Railway.

smoky creek Oct 22, 2023, 6:21 AM

#

just because something doesn't work on Railway doesn't always mean railway is at fault, I've setup a few php based apps with volumes mounted to /var/www/html without issues

grand sage Oct 25, 2023, 7:18 PM

#

I've looked at this some more and while I can't say for certain the fault lies with Railway, I'm at a loss for what else could be the issue.

Based on the mediawiki Dockerfile source used to generate their image
https://github.com/wikimedia/mediawiki-docker/blob/fdf347f62615dad789d0e703fc2f6d628c43d4e1/1.40/apache/Dockerfile
I've broken things down to the simplest case that results in the service not running as expected:


RUN set -eux; \
    { \
        echo "<html>"; \
        echo "  <body><p>Hello world</p></body>"; \
        echo "</body>"; \
    } > "index.html"

CMD ["apache2-foreground"]

Using that custom Dockerfile in a new service, I am able to successfully run this on Railway and accessing the generated domain gives me the expected 'Hello world' page.

However, mounting a volume to /var/www/html on that same service then trying to run with no other changes, I receive a 403-Forbidden error along with this entry in my my Deploy logs on Railway:
AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.php,index.html) found, and server-generated directory index forbidden by Options directive

smoky creek Oct 25, 2023, 7:20 PM

#

attaching a volume to a mount point that already has files will resulting in that folder having no files since the volume is empty

grand sage Oct 25, 2023, 7:21 PM

#

This also occurs when mounting the volume before building the image

smoky creek Oct 25, 2023, 7:21 PM

#

of course

grand sage Oct 25, 2023, 7:22 PM

#

Although, incidentally, I thought that mounting a volume within images would preserve the pre-existing contents.

smoky creek Oct 25, 2023, 7:22 PM

#

nope, this is a drive mount

#

just a regular ext4 drive mount

#

and this is the only time I've seen this type of thing cause an issue, so while it may seem lazy on my part, I'm chalking this up to mediawiki being poorly designed

grand sage Oct 25, 2023, 7:24 PM

#

Ok, but treating it as such, what's the proper approach to mount and then have the mounted directory's contents affected by my image?

#

Because the dockerfile above has nothing to do with mediawiki - it should, I would think, work with a volume attached,

smoky creek Oct 25, 2023, 7:25 PM

#

mediawiki should not be asking you to mount to a location that already has files, the reason I'm calling this poor design on their part

grand sage Oct 25, 2023, 7:26 PM

#

Fair, but returning to the dockerfile example above - is there a work-around to allow index.html be written into the mounted /var/www/html directory, as intended?

smoky creek Oct 25, 2023, 7:26 PM

#

you would want to write it after the container start

grand sage Oct 25, 2023, 7:27 PM

#

So there is no way to access volumes in Railway during the build process?

#

I guess that's the wrong question, really.

smoky creek Oct 25, 2023, 7:28 PM

#

that is correct, the volume is not available during build

grand sage Oct 25, 2023, 7:29 PM

#

At least comparing (naively) Railway to running a Docker image on my own PC, this is due to a fundamental difference in how volumes are handled by Docker by default vs how Railway is implementing volumes.

smoky creek Oct 25, 2023, 7:30 PM

#

yes there is a difference, but if mediawiki was properly designed this would not be an issue

grand sage Oct 25, 2023, 7:30 PM

#

I'm interacting with dockerfiles and expecting the volume stuff to be the same as with Docker, but Railway is actually doing other work like converting to nixpacks (iirc) and handling volumes mounted to services in an unrelated way.

smoky creek Oct 25, 2023, 7:31 PM

#

mediawiki is the only app I've seen have this issue

grand sage Oct 25, 2023, 7:32 PM

#

Yeah I hear you. I'm more thinking of my own learning though because I'm seeing now I had a misapprehension about what was happening with volumes. The MediaWiki design is fair to criticize, but I presume it's likely because it came about in an era before distributed deploys like docker/railway/etc.

smoky creek Oct 25, 2023, 7:35 PM

#

yeah it was probably never really designed with this kind of hosting anywhere in mind, after all they want you to mount a volume to a file path instead of a folder path, that's yikes to say the least

grand sage Oct 26, 2023, 2:13 AM

#

Alright, I'm looking for a sanity check that this isn't overly stupid or otherwise A Real Bad Idea. It's possibly convoluted, but I can accept that. 😄

#

I'm interested in MediaWiki despite its design flaws (as well as learning a bit about Docker and Railway along the way) which means I need to be able to access /var/www/html on my volume. A mounted volume in Railway can only be accessed by the app after the container is running. With the default MediaWiki DockerHub image, I note that the actual MediaWiki "install" portion basically amounts to downloading a tarball using curl and extracting it into /var/www/html, something that doesn't work if that's where the volume is mounted.

My work-around is to use a custom Dockerfile hosted on a private repo that is a copy of the MediaWiki file with a few key lines added. The end result is that the MW files end up untarred into a temp directory, and an executable shell script is used as the main CMD entrypoint instead of running apache. This shell script first copies all of the MW files from the temp directory into /var/www/html (which will now be the mounted volume, at "runtime" for the container), deletes the temp directory, then runs the apache server as per usual.

smoky creek Oct 26, 2023, 2:18 AM

#

yeah that could work

grand sage Oct 26, 2023, 2:25 AM

#

Alright, fewf. It does seem to at least be running and using your FileBrowser template additionally allowed copying the LocalSettings file over to the volume. It’s clumsy, but it may work.
Many thanks for your help!!

#Volume storage not available while building from Docker image?