Set up shared Julia on server

[small onyx]

Folks, I've recently taken on the responsibility to rebuild an elderly shared server that is used to run bioinformatic data pipelines, some of which are written in Julia.

I'm having a challenge that some of the packages I need are no longer available. They do exist on the old machine at .julia/packages/ThePackage, but I've been unsuccessful at pulling those out and adding them to the install on the new machine. Any suggestions?

I'm a total Julia beginner, but have a lot of linux and development experience.

[wary bronze]

So you mean you have the installed packages, but not the source code?

Also, just as a resource I've appreciated, this video is a good crash course on setting up your Julia development environment
https://youtu.be/-lJK92bEKow

[small onyx]

I think I have the source, but I'm not entirely sure. There are .jl files, which I think are the source? The directory I'm looking at looks like this:

└── 9Lc74
    ├── LICENSE.md
    ├── Manifest.toml
    ├── Project.toml
    ├── README.md
    ├── REQUIRE
    ├── appveyor.yml
    ├── docs
    │   ├── build
    │   │   ├── assets
    │   │   │   ├── Documenter.css
    │   │   │   └── mathjaxhelper.js
    │   │   ├── dp.md
    │   │   └── index.md
    │   ├── make.jl
    │   ├── mkdocs.yml
    │   └── src
    │       ├── dp.md
    │       └── index.md
    ├── src
    │   ├── DPMeansClustering.jl
    │   ├── dp.jl
    │   └── include_all.jl
    └── test
        ├── runtests.jl
        └── test_dp.jl

[wary bronze]

Oh yeah that looks like source code.

[balmy prawn]

Julia generally would not just have object code, it's pretty much always source, sometimes (in more recent versions) compiled object code cached as well.

Also, any packages (at least that are public) should be downloadable on the new machine, you shouldn't need to transfer them from the old machine

For example, that one is at https://github.com/MurrellGroup/DPMeansClustering.jl

[wary bronze]

Yeah, if you have internal package then you'll probably want to set up an internal package registry. But, otherwise, you'll just want to take advantage of Julia's Pkg mode and add your dependencies to each project as they're required

[small onyx]

I swear that I searched, several times, for that! Now I feel stupid.

It's not clear to me when I'm adding packages to Julia, if they are just being installed for my user, or if they're available machine wide?

[balmy prawn]

Generally, they are just added for that user (in the ~/.julia/packages directory) - and different versions may be copied there as well (if you are running different julia versions)

For example, I have 6 different versions of my Format package cached:

(base) ✔ ~/.julia/packages/Format
12:22 $ ls -ltr
total 0
drwxr-xr-x  11 scott  staff  352 Oct 11  2021 D5BQM
drwxr-xr-x  10 scott  staff  320 Feb 25  2022 K3u3F
drwxr-xr-x  11 scott  staff  352 Feb  1 09:51 6kJWb
drwxr-xr-x@ 11 scott  staff  352 Feb 15 21:46 ssMbK
drwxr-xr-x@ 11 scott  staff  352 Mar 12 11:27 js0Sa
drwxr-xr-x  11 scott  staff  352 Mar 27 11:18 0LfJN

[wary bronze]

If you don't activate a project (typically done via the REPL in Pkg mode as ] activate .), then it'll be installed in the current Julia version.

Pkg mode is super useful, it's a lot like having PyPI built into the language (if you're familiar with Python)

[small onyx]

That's what I thought, but it's good to have confirmed.
I've been using Pkg mode a ton. 🙂