#how do i install packages?

251 messages · Page 1 of 1 (latest)

prisma dagger
analog fernBOT
#

This post has been reserved for your question.

Hey @prisma dagger! Please use /close or the Close Post button above when you're finished. Please remember to follow the help guidelines. This post will be automatically closed after 300 minutes of inactivity.

TIP: Narrow down your issue to simple and precise questions to maximize the chance that others will reply in here.

prisma dagger
knotty dove
# prisma dagger no

download the latest release under the release tag and add it as a library in your favorite ide

prisma dagger
#

i dont see

knotty dove
#

It appears using this requires further configuration. Check the readme of the repository.

prisma dagger
#

i did

#

i cant find the jar

finite jolt
#

DL4J itself has multiple dependencies

#

Using a build tool like Maven or Gradle and not downloading the JAR will be easier

#

especially since it also uses native components (I think)

#

Deeplearning4J has quite a few dependencies. For this reason we only support usage with a build tool.
Yeah, you should use Maven or Gradle

finite jolt
#

What's the problem with Maven?

prisma dagger
#

C:\Users\fried>mvn --version
The JAVA_HOME environment variable is not defined correctly,
this environment variable is needed to run this program.

finite jolt
#

What's the output of javac -version?

prisma dagger
#
javac 19.0.2
finite jolt
#

What's the output of where javac?

prisma dagger
#

C:\Program Files\Common Files\Oracle\Java\javapath\javac.exe

finite jolt
#

You should have a program called Edit environment variables ir similar on your computer.

prisma dagger
#

it says JAVA_HOME is at C:\Program Files\Java\jdk-19

finite jolt
#

what happens if you enter echo %JAVA_HOME% in cmd?

prisma dagger
#

"C:\Java\jdk-18.0.2.1_windows-x64_bin.exe"

finite jolt
#

ok that's wrong

#

Did you change the JAVA_HOME?

prisma dagger
#

?

finite jolt
#

typo, fixed

prisma dagger
#

did i change it to what

finite jolt
prisma dagger
#

yes

finite jolt
#

Did you restart cmd after changing it?

prisma dagger
finite jolt
prisma dagger
#

restarting cmd worked

finite jolt
prisma dagger
#

C:\Users\fried>echo %JAVA_HOME%
C:\Program Files\Java\jdk-19

finite jolt
prisma dagger
#

'mvn' is not recognized as an internal or external command,
operable program or batch file.

#

also the fact i dont know how

finite jolt
#

well you need to set the PATH to maven

prisma dagger
#

i did

finite jolt
#

that is reset/reloaded when restarting cmd

#

just like JAVA_HOME or other environment variables

prisma dagger
#

what

finite jolt
#

if you use set PATH=..., that will only be valid for your cmd session

prisma dagger
#

no i set it in environment variables

finite jolt
#

but you don't necessarily need to use the mvn command line for using Maven

prisma dagger
#

i want mvn to work

finite jolt
prisma dagger
#

C:\Users\fried>echo %PATH%
C:\Program Files\Common Files\Oracle\Java\javapath;C:\Program Files (x86)\VMware\VMware Player\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Program Files\dotnet;C:\Users\fried.cargo\bin;C:\Users\fried.cargo\bin;C:\Users\fried\AppData\Local\Programs\Python\Python310\Scripts;C:\Users\fried\AppData\Local\Programs\Python\Python310;C:\Users\fried\AppData\Local\Programs\Python\Python311\Scripts;C:\Users\fried\AppData\Local\Programs\Python\Python311;C:\Users\fried\AppData\Local\Microsoft\WindowsApps;;C:\Users\fried\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\fried.dotnet\tools;C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2022.3.1\bin;

C:\Users\fried>

finite jolt
#

I don't see Maven in there

#

Did you restart cmd after adding Maven to your PATH?

prisma dagger
#

ys

finite jolt
#

How did you add it to your PATH?

prisma dagger
#

System variables

MAVEN_HOME
C:\Program Files\Java\apache-maven-3.9.1

finite jolt
#

oh that's the problem

#

you need to edit the PATH environment variable

#

and add C:\Program Files\Java\apache-maven-3.9.1\bin to it

prisma dagger
#

what is path

#

C:\Users\fried>mvn --version
Apache Maven 3.9.1 (2e178502fcdbffc201671fb2537d0cb4b4cc58f8)
Maven home: C:\Program Files\Java\apache-maven-3.9.1
Java version: 19.0.2, vendor: Oracle Corporation, runtime: C:\Program Files\Java\jdk-19
Default locale: en_US, platform encoding: UTF-8
OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"

finite jolt
#

it's an environment variable

finite jolt
#

Now you can set up your Maven project

prisma dagger
# finite jolt Now you can set up your Maven project

PS C:\Users\fried\Desktop\Saratoga-MK3> mvn give me Nd4j
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.064 s
[INFO] Finished at: 2023-04-18T17:08:22-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] The goal you specified requires a project to execute but there is no POM in this directory (C:\Users\fried\Desktop\Saratoga-MK3). Please verify you invoked Maven from the correct directory. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MissingProjectException
PS C:\Users\fried\Desktop\Saratoga-MK3>

finite jolt
#

it doesn't work like that

#

First, you need to create a Maven project

prisma dagger
#

how do

finite jolt
#

What IDE are you using?

prisma dagger
#

vsc

finite jolt
#

oh no

#

Do you have some option to create a Maven project in VSC?

prisma dagger
#

idk

#

i hate visual studio and intellij

finite jolt
prisma dagger
#

i dont like vs

#

is bad

finite jolt
#

this link is for VSC

prisma dagger
#

oh

#

how do i create it on existing project

finite jolt
#

just create a new project

#

I have no idea about VSC

prisma dagger
finite jolt
prisma dagger
#

there isnt an option in vsc to make a new project

#

it doesnt really use projects

#

maven is stoopit

finite jolt
#

this isn't really a Maven problem

#

more a VSC problem

finite jolt
#

make sure you have the extension

analog fernBOT
#

💤 Post marked as dormant

This post has been inactive for over 300 minutes, thus, it has been archived.
If your question was not answered yet, feel free to re-open this post or create a new one.

prisma dagger
#

my god i hate everything about intelij

#

the colors are bad

#

the font is bad

#

there is a weird line that i dont like

#

it created hundreds of folders and files that i dont want and i dont know what they are

#

it fails to do anything related to dependancies

#

and i have no fucking idea how it is considered better than vsc

pastel gate
#

Well for starters it doesn't fail anything regarding dependencies

prisma dagger
pastel gate
#

And if we are to believe you, vsc has no notion of projects. That means that any pile of shit that does handle projects, is all the reason you need to drop vsc forever and use the pile of shit instead

pastel gate
pastel gate
#

Am I supposed to see anything? Looks like your pom is bad or something

prisma dagger
#
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.example</groupId>
  <artifactId>Saratoga</artifactId>
  <version>1.0-SNAPSHOT</version>
  <name>Archetype - Saratoga</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>org.nd4j</groupId>
      <artifactId>nd4j-native</artifactId>
      <version>1.0.0-M2.1</version>
    </dependency>
    <dependency>
      <groupId>org.deeplearning4j</groupId>
      <artifactId>deeplearning4j-core</artifactId>
      <version>1.0.0-M2.1</version>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <version>1.4.6</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>16</source>
          <target>16</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>
pastel gate
#

It declares artifacts with dependencies that seem to not exist, apparently

prisma dagger
#

where

pastel gate
#

How would I know, I'd need the logs

prisma dagger
#

where logs

pastel gate
#

Where they're supposed to be

prisma dagger
#

where logs

pastel gate
#

At all places that aren't where they're not supposed to be

prisma dagger
#

where logs

finite jolt
#

Press Shift twice

#

a text box should pop up

#

then enter mvn dependency:go-offline install

#

this should download all dependencies

#

IntelliJ sometimes lies about dependencies not existing if it can't find them locally

finite jolt
analog fernBOT
#

💤 Post marked as dormant

This post has been inactive for over 300 minutes, thus, it has been archived.
If your question was not answered yet, feel free to re-open this post or create a new one.

knotty dove
#

A Run Anything prompt should appear

prisma dagger
#

build success

knotty dove
#

All dependencies can be resolved ?

finite jolt
finite jolt
finite jolt
#

so now you downloaded them

prisma dagger
#

still red

finite jolt
#

and you can tell IJ to check again

prisma dagger
#

wat

finite jolt
#

Did you use that button?

prisma dagger
#

oh cool

#

what do these do?

finite jolt
finite jolt
prisma dagger
#

are any important

finite jolt
#

e.g. clean deletes the target folder, compile tells Maven to compile the project, test tells maven to run your tests, install tells Maven to build the project and save it to the local repository etc.

prisma dagger
#

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

finite jolt
#

maybe try adding slf4j-api to your pom.xml

prisma dagger
#

is there a command for that or do i do it by hand

finite jolt
#
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api<artifactId>
    <version>YOUR-VERSION</version>
</dependency>
finite jolt
#

the version of SLF4J-API

prisma dagger
#

where do i find that

finite jolt
#

maybe IntelliJ suggests it to you

#

the newest is 2.0.7

prisma dagger
#

Dependency 'org.slf4j:slf4j-api:2.0.7' not found

finite jolt
#

same as previously

#

download dependencies, reload project

prisma dagger
#

maven install right?

finite jolt
#

yeah

prisma dagger
#

that wasnt so bad

#
14:34:47.049 [pool-1-thread-5] INFO org.nd4j.nativeblas.NativeOpsHolder -- Number of threads used for linear algebra: 6
14:34:47.052 [pool-1-thread-5] INFO org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory -- Binary level Generic x86 optimization level AVX/AVX2
14:34:47.071 [pool-1-thread-5] INFO org.nd4j.nativeblas.Nd4jBlas -- Number of threads used for OpenMP BLAS: 6
14:34:47.088 [pool-1-thread-5] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner -- Backend used: [CPU]; OS: [Windows 11]
14:34:47.088 [pool-1-thread-5] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner -- Cores: [12]; Memory: [12.0GB];
14:34:47.088 [pool-1-thread-5] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner -- Blas vendor: [OPENBLAS]
14:34:47.091 [pool-1-thread-5] INFO org.nd4j.linalg.cpu.nativecpu.CpuBackend -- Backend build information:
 GCC: "12.1.0"
STD version: 201103L
DEFAULT_ENGINE: samediff::ENGINE_CPU
HAVE_FLATBUFFERS
HAVE_OPENBLAS```
#

i assume this means it worked

finite jolt
#

yes

prisma dagger
#

and it prints that every time i run?

finite jolt
#

it should be possible to disable it somehow

prisma dagger
#

also why does it say CPU instead of GPU

finite jolt
#

because you used the CPU dependency?

prisma dagger
#

where did i do that

finite jolt
#

in the pom.xml

#

nd4j-native is cpu I think

prisma dagger
#
        double[][][] vectorB = matrixB.convertToVector();
        double[][][] vectorA = convertToVector();
        INDArray tensorB = Nd4j.create(vectorB);
        INDArray tensorA = Nd4j.create(vectorA);
        INDArray result = tensorA.mmul(tensorB);
        return convertToMatrix(result);```
#
    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
    at Main.train(Main.java:95)
    at Main.main(Main.java:39)
Caused by: java.lang.IllegalStateException: Rank is 3 rows() call is not valid
    at org.nd4j.linalg.api.ndarray.BaseNDArray.rows(BaseNDArray.java:4069)
    at org.nd4j.linalg.api.ndarray.BaseNDArray.mmul(BaseNDArray.java:2913)
    at org.nd4j.linalg.api.ndarray.BaseNDArray.mmul(BaseNDArray.java:2922)
    at Matrix.multiply(Matrix.java:308)
    at Softmax.backward(Softmax.java:36)
    at Model.backward(Model.java:32)
    at TrainingTask.trainImages(TrainingTask.java:25)
    at TrainingTask.call(TrainingTask.java:36)
    at TrainingTask.call(TrainingTask.java:3)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1589)

Process finished with exit code 1
finite jolt
#

what are the matrix dimensions?

prisma dagger
#

why is the debug console so shit

#

it doesnt save the result of previous commands

#

tensorA -> [1,2,2]
tensorB -> [1,2,1]

finite jolt
#

are these the values or the dimensions?

prisma dagger
#

dims

#

values are random doubles

finite jolt
#

so you are trying to do matrix multiplication on 3D matrices?

prisma dagger
#

but both have a z level of 1

#

so it should essentially be a 2D multiplication

#

not that 3D even is an issue

finite jolt
#

Can you try to resize them to 2D matrices?

#

before doibg the matrix multiplication

prisma dagger
#

The whole point of the import is for n dimensional matrices

#

Why can’t it do a 3d multiply

finite jolt
#

try 2D and check whether it works

#

Matrix multiplication isn't generally defined for 3D matrices - why do you think it would work?

#

can you multiply two 1x1x1 matrices together?

finite jolt
prisma dagger
#

It is defined

finite jolt
#

by what?

prisma dagger
#

math

finite jolt
#

Are you sure there's a universally agreed definition of it?

prisma dagger
#

yes

#

Einstein created a whole notation for high dimension tensor operations

finite jolt
#

Also on what is the dimension which should be "ignored"?

prisma dagger
#

none

finite jolt
#

So what's your formula for 3D matrix multiplication?

prisma dagger
#

C(i, j, k) = ∑[A(i, j, m) * B(m, k, n)]

#

this uses the einstein notation

#

Matrix A: (p x q x r)
Matrix B: (q x r x s)
Matrix C: (p x r x s)

finite jolt
#

where does m and n come from?

#

these areboth not present on the left side

#

are both indices defined by the sum?

prisma dagger
#

The variables "m" and "n" represent the indices along the second and fourth dimensions of matrices A and B, respectively.

finite jolt
#

(btw you can use LaTeX here if you are using two dollar signs before and after)

prisma dagger
#

idk how to latex

finite jolt
#

so you mean there are two sums?

prisma dagger
#

idk how to do it

#

i know know its defined

#

HOLY CRAP IT WORKS

#

BLAS assembly cut my run time in half

finite jolt
#

so basically $$C_{i,j,k}=\sum_{m=1}^{M}\sum_{n=1}^{N} A_{i,j,m}B_{m,n,k}$$?

pine ridgeBOT
prisma dagger
#

shore

#

but that would mean the z level of 1 matrix cant equal the z level of the second matrix

#

which is why 1xnxm * 1xmxn doesnt work

finite jolt
# pine ridge

if that is the formula, the inner_most_ dimensions must match

#

With 2 dimensions, it is clear that the inner dimensions must match but with 3D - dimensions could mean different things

#

e.g. one dimension could be for parallelization or whatever

#

in which case there would be two dimensions which would need to match

prisma dagger
#

i converted them to 2D and at this spot in my code i dont think the z level will ever be greater than 1

finite jolt
#

I don't know what exactly you mean with z level

#

one of the problems of mathematics is that different people use different terms and definitions

#

paradoxically, mathematicians are really bad at agreeing on notation

finite jolt
# pine ridge

like that's one possibility of defining it but you could also say something like $$C_{i,j,k}=\sum_{l=1}^L A_{i,j,l}B_{i,l,k}$$

pine ridgeBOT
prisma dagger
#

i wonder if it could be defined using the capitol PI notation using purely sub dot products

#

thats the way i had it programmed in my code prior to BLAS library

finite jolt
#

that would require you having large products

#

I mean products over an index

prisma dagger
#

why does the intelij debug console not save previous command results in the terminal

finite jolt
#

because it doesn't

#

you can't just assume programs do what you think they do

prisma dagger
finite jolt
#

so what?

#

That's no reason to assume another program does the same

prisma dagger
#

smh

#

right when i was starting to think intilij was decent

#

doesnt even have a proper debugger

analog fernBOT
#

💤 Post marked as dormant

This post has been inactive for over 300 minutes, thus, it has been archived.
If your question was not answered yet, feel free to re-open this post or create a new one.

finite jolt
#

IJ does have a proper debugger