Page Files, how do they work?

[mortal shoal]

Here

[radiant tulip]

Ah good idea

[torpid vessel]

hope the convo's over.

[bright sun]

[ionic river]

[torpid vessel]

I really feel like there's no reason to continue this discussion. the hinge of the argument is literally computing 101.

[karmic parcel]

page files work by voodoo magic and I refuse to accept any other answer

[timid cargo]

how does this got upgrade to a thread

[radiant tulip]

Hey it's a thread so you don't have to care about it

[real viper]

how do you page a file??

[abstract granite]

page files are when the files get paged

[ionic river]

Page files, also known as swap files or virtual memory files, are an integral part of modern computer operating systems, designed to efficiently manage the allocation of physical RAM (Random Access Memory) and secondary storage, typically a hard drive or SSD. They play a crucial role in extending a computer's available memory beyond its physical limitations. When a computer runs multiple applications simultaneously and the physical RAM becomes fully or nearly occupied, the operating system utilizes page files to store portions of data that are currently inactive or less frequently accessed. These data segments, often called pages or virtual memory pages, are swapped in and out of RAM as needed. The page file contains these pages, acting as a supplement to physical memory. When a program or process requires more memory than is available in RAM, the operating system transfers the least used pages from RAM to the page file, freeing up space for the new data. Conversely, when a program needs data stored in the page file, it's swapped back into RAM, allowing the system to operate seamlessly. This process, called paging, helps prevent system crashes due to insufficient memory and enables multitasking and efficient memory management. Page files are a dynamic and vital component of modern computing, providing a balance between system performance and available memory.

[torpid vessel]

it's a very multifaceted approach that solves many, many problems at once

[real viper]

what if i store everything in ram

[fair marsh]

not everyone is a computer science graduate for the record, I have no idea how it works at low level, but I know that RAM is faster than SSD. If CPU can access the SSD, read from it, and do whatever it has to do faster than it can do on RAM then you're correct, but I can't find this info online

[real viper]

1tb of ram

[torpid vessel]

im going to stab you.

[radiant tulip]

You're somehow saying ram can't serve as cache because it has to lookup on the ssd at least once or something

That doesn't make any sense

[proper shoal]

You're online

[radiant tulip]

Page files may not even be used if you have sufficient ram

[karmic parcel]

it still has to pull it back into RAM from the SSD before the CPU can access it

[torpid vessel]

no, im saying ram is terrible at being the only cache.

[real viper]

how to ram persistent storage in ram?

[torpid vessel]

i have nothing against ram caching.

i am not defending paging as the only caching method

[radiant tulip]

You're saying I'm going against that point though it felt like to me

[torpid vessel]

i am defending paging as an existing caching method when the question is why it exists.

no...

I'm saying eliminating paging files will not be faster than ram caching, just inherently.

[real viper]

i think my page file is disabled

[radiant tulip]

I mean that's just false

[real viper]

i never use over 16gb though

[radiant tulip]

Inherently every process and its associated pages are in ram

It's only when not efficiently enough space is left in ram you would need to use a page file

[torpid vessel]

...no, what?

[radiant tulip]

That's how main memory works

[torpid vessel]

no, page files always run simultaneously with physical memory.

it's not "one or the other".

all x86 computers use both physical ram and page files at the same time.

[karmic parcel]

this is dumb

[fair marsh]

I think we are not on the same page (pun not intended)

[torpid vessel]

if one is filled, the other is used as an only alternative.

[karmic parcel]

if you have 16gb you definitely need a page file on

[fair marsh]

So, you said this:

[karmic parcel]

I have seen games allocate over 20gb

[real viper]

i don’t remember

[fair marsh]

I read that as "why would you add ram if you can just go from CPU to storage"

[radiant tulip]

Generally a process that's sleeping or not currently scheduled is "frozen" and then put into a page file, that's why the page file exists. Do you disagree? @torpid vessel

[fair marsh]

well, let's say you have 1GB cached data. My point is you can read that 1GB form RAM faster than you do on disk.

[torpid vessel]

Yes, why would you make the caching be in the ram when it can just be in a page file

[radiant tulip]

That occurs when there isn't sufficient ram

[radiant tulip]

Because ram is faster

[fair marsh]

I have 48GB of RAM at this point, why would I want to use the SSD

[torpid vessel]

Yeah, I disagree, that's not how things happen.

[radiant tulip]

What do you suggest the page file is for then

It's not faster than ram it's a method that's used because ram may be insufficient

[karmic parcel]

cause it allows you to have more crap open

[torpid vessel]

For storing processes that do not need to be in the ram. Not for the sake of having sufficient ram, but for the sake of having optimized access.

[fair marsh]

I literally said that the only downside is, if you run out of RAM, your apps start crashing. It is true however that some apps force data into the paging file for some reason I don't understand, either obsolete programming or you're working with RAM and can't guarantee it will not be deleted, in those scenarios you do need a paging file.

[radiant tulip]

Optimized access is ram

[torpid vessel]

...no

[fair marsh]

yeah that's a good point, but I'm trying to make sure here that I don't ever fill it up to the rim among other things

[radiant tulip]

No ok optimized access are cpu registers

Then cache

Then ram

[torpid vessel]

werdna, you have a computer engineering masters, right

you feel like weighing in

[karmic parcel]

I have a BS not a masters

[torpid vessel]

ah.

[radiant tulip]

I don't understand why you feel a page file is necessary when ram won't run out

On a normal desktop work station there isn't a point

[timid cargo]

Untrue. Caching of background and frequently used apps in Windows management (I cant speak for Linux and other OS) is not black and white. Both active, unused memory and page files are used in the process.

[karmic parcel]

because if you are not running out, there is no downside to having it enabled

it should never swap to it

[radiant tulip]

I never said there was

It's just useless

[torpid vessel]

Cause a page file allows for the optimal transfer of information both between the ssd and the ram. not for the sake of capacity.

[fair marsh]

that was my question because I was hit with "bro this is so computer science 101". I legit would like to be educated if I'm wrong.

[karmic parcel]

except it isnt because you may still end up with more than 48gb allocated

[torpid vessel]

there are more things to worry about in memory optimization than capacity.

[torpid vessel]

and like I said, please read any article on virtual ram.
I'm genuinely not trying to be condescending. I don't understand what's not clicking, cause this is something google can solve.

I'm not saying "google it" to be snarky, im saying "google it" cause it's the simplest way for you to figure out what virtual ram does.

[fair marsh]

If I may add, the initial scenario was disregarding sizes. JKIN was claiming paging file was straight faster than RAM because that's how it "was supposed to work since x86"

[karmic parcel]

people also suck at measuring actual RAM usage

[radiant tulip]

So when you load from an ssd, you load into ram and ram is divided into pages. These pages are generally 4kib big and have associated processes or belong to the kernel. These are what go to the page file, there's nothing optimized about it compared to filling ram as much as possible. @torpid vessel

[karmic parcel]

cause task manager lies

[karmic parcel]

OK well a page file is definitely not faster than RAM and I dont think that is what he was claiming

[fair marsh]

I swear I've hit 5 articles, all the downsides I see on paging file are just specific apps who hit the paging file on demand, and running out of RAM, everything else is worse according to Google.

[torpid vessel]

I'm claiming eliminating the page file makes the overall compute progress slower than with just ram.
Not saying that paging files are slower than ram.

[radiant tulip]

I am saying they're slower as they must be put back into ram

[torpid vessel]

Except that's not how anything commits by.

[radiant tulip]

That's why you use them only when needed

[fair marsh]

maybe I read that wrong

[karmic parcel]

could just be *nix and kill programs when you run out

[torpid vessel]

I'm literally saying the ssd would be dragging down the ram anyway, so why make the ssd slower

[fair marsh]

but what I get from that pic is "Now you made it so it's the ssd, ram, and cpu. Quite slower"

[karmic parcel]

this sounds like a comment about loading assets from the drive which is unrelated to swap?

[torpid vessel]

we love page file discussion.

[radiant tulip]

I don't understand the point being made here. Ram is a faster 'cache' than an ssd and it's what the cpu operates on. There's no need for a page file unless you're going to expect the os to hoard all your ram

[torpid vessel]

you do realize nothing in x86 computing goes by that standard, right

[radiant tulip]

What standard

[torpid vessel]

if that were true, yes, there would be no reason to have a paging file.

the standard of filling physical ram cause it's inconsequential

[radiant tulip]

When you say cache here do you mean on chip cache like l1/l2?

Or ram

[torpid vessel]

I mean physical ram.

but sure, we can go with cache on the chip as well.

[radiant tulip]

That's not x86 related at all

That's nothing to do with cpu architecture

That's purely os related

[torpid vessel]

...thus x86

[radiant tulip]

Lmfao

What

[torpid vessel]

x86 is not a cpu architecture

[radiant tulip]

It is a family of architectures

Primarily x86-64

[torpid vessel]

designed around a specific compute protocol for os interaction.

[radiant tulip]

Which you're using right now

[torpid vessel]

I'm just using x86 as an example, cause there are some compute protocols that do not use paging files.

[radiant tulip]

Nothing like that relates to ram usage

[karmic parcel]

how do you know I am not on my phone

[fair marsh]

Now I have no clue what I'm talking about, all I know is x64 uses a larger instruction set, boom.

[torpid vessel]

I really don't know how to continue this conversation if your new claim is that x86 doesn't influence ram behavior.

[radiant tulip]

Ok hypothetically if a machine had Infinite ram would it still make use of paging files? @torpid vessel

[radiant tulip]

It doesn't really

Not much

[torpid vessel]

yes, because that's what it'd be programmed to do.

[radiant tulip]

There's nothing x86 inherent with page files

It's an os detail

[karmic parcel]

imagine thinking you are smarter than the engineers at microsoft

[torpid vessel]

x86 literally has a defined mmu

what do you mean x86 has nothing inherent with page files

x86 might as well be a blueprint for caching

[radiant tulip]

The mmu relates to paging on main memory or in other words ram

The page file is a reflection of that

[torpid vessel]

the mmu relates to all caching.

[radiant tulip]

It's not necessitated by the architecture

[torpid vessel]

It's not. Tell that to Windows or Linux.

[radiant tulip]

That's not really accurate but ok, if you call ssd a cache then absolutely not

[torpid vessel]

...are we not discussing paging files on ssds

that's literally the primary topic of the discussion.
in this context, yes, the ssd is being used as a cache.

[radiant tulip]

How does the mmu relate to needing a page file? Or what are you claiming

[karmic parcel]

Sometimes, a page table entry or other per-page information prohibits access to a particular virtual page, perhaps because no physical random-access memory (RAM) has been allocated to that virtual page. In this case, the MMU signals a page fault to the CPU. The operating system (OS) then handles the situation, perhaps by trying to find a spare frame of RAM and set up the page map to map it to the requested virtual address. If no RAM is free, it may be necessary to choose an existing page (known as a victim), using some replacement algorithm, and save it to disk (a process called paging). With some MMUs, there can also be a shortage of PTEs, in which case the OS will have to free one for the new mapping.[3]

the MMU calls for the file to be paged to disk

[radiant tulip]

A page fault isn't what you think it is

Physical memory is ram

It's what happens when a virtual address is unmapped

Nothing to do with the ssd

[torpid vessel]

I brought up x86 to give a bound to "all compute"
I would refine this further with how windows and linux interact with x86-based systems.

which very much requires ssd paging for optimal performance.

[radiant tulip]

Your claim seems to be x86 is somehow design to care for a page file when it's not

The ssd is a peripheral, it's io

[torpid vessel]

Again, I brought up x86 to narrow the context of this discussion.

[radiant tulip]

To what?

[karmic parcel]

If no RAM is free, it may be necessary to choose an existing page (known as a victim), using some replacement algorithm, and save it to disk (a process called paging).

[torpid vessel]

not to link it to page files. just to view the discussion in the perspective of x86

[karmic parcel]

what does that say

[torpid vessel]

to further narrow the context of this discussion, the x86 system would be utilizing either linux or windows.

[radiant tulip]

You realize that it says "may" and do you realize that's a feature of the os, not the cpu hence not x86?

[torpid vessel]

both benefit from page files being on.

I don't know why you're stuck on x86. I brought it up to narrow the range of computing behavior in active discussion.

[karmic parcel]

the may is because it may find another way to free RAM such as compression

[torpid vessel]

no one is linking page files to be a necessity in x86, just that x86 has inherent protocol for page files that can be used when present.

[radiant tulip]

The "may" can also indicate you don't run out of ram. The only way you would is by memory mapping files which the os generally does but these have no point being put into a page file rather than flushed back into their respective files.

[torpid vessel]

and yes, I've been trying to move on to OS level discussion for the past 10 minutes.

storage device paging is very much a necessity for both windows and linux for optimal performance.

[radiant tulip]

You're the one mentioning x86 to me originally which is completely redundant to this discussion really.

[torpid vessel]

I've explained why I brought it up so many times.

I know it's redundant. this whole conversation is.

I don't get why this is an argument at all, I haven't from the beginning.

Feels pretty simple to me.
"Windows and Linux require storage paging to be enabled for optimal performance. turning off storage paging will give an inherent performance decrease relative to having it on."

[radiant tulip]

I said why it's unrelated. You mention a protocol? Intel/amd engineers don't create protocols for os developers save for a few tools to accomplish cpu specific things. It's meaningless to bring up x86 in a discussion about operating systems and peripherals

[radiant tulip]

That's not even true there's nothing optimal about it unless your ram is crap.

[torpid vessel]

x86 is a protocol.

[radiant tulip]

It's not

[torpid vessel]

you know what, I'm dropping it here, we still aren't on the same page, I'm not putting more time into this when we're arguing on baseline.

[radiant tulip]

Fair enough

Agree to disagree

[torpid vessel]

No hard feelings, though im just tired and rambly

[radiant tulip]

Sure

For what it's worth

I attempted to program bare metal x86 and developed a fun os on it as a toy project reaching barebones userspace. It's nothing too impressive but that's why I'm passionate about this discussion and what you're saying.

I'm by no means a master but it's a hobby of mine

[torpid vessel]

Oh, that's cool

[radiant tulip]

So when you're telling me x86 is a protocol and pagefiles are needed it indicates to me that you don't have a similar experience with x86 as me and so we're operating from different viewpoints

Which my perspective could be deluded as it's mostly programming related

And so that's the front I operate from

[torpid vessel]

Yeah, I was discussing in the context of windows, which does have firmware-level optimization even on the hardware platform end.

I do firmware design for agesa, so that's my front. very different from yours.

[radiant tulip]

Yes indeed