VASP Forum

I'm doing a nonlinear calculation on a slab geometry with 120 ions. My goal is to determine the band structure but I have been running into memory issues. To better understand what is going on I want to estimate the memory which will be used like described in https://vasp.at/wiki/Memory_requirements. From what I understand these values can be found in the OUTCAR file.

When I look at my OUTCAR file it describes these values except NRPLWV. I assume that this is the same as the value NPLWV, which in my OUTCAR file shows up as ******. I assume this happens due to the large number of plane waves within my calculation. Would there be a way to retrieve this value in some other way or to dispaly it in the OUTCAR file anyway? Or is there another better way of estimating the memory usage?

part of the OUTCAR file:

k-points           NKPTS =     80   k-points in BZ     NKDIM =     80   number of bands    NBANDS=   1328
   number of dos      NEDOS =    301   number of ions     NIONS =    120
   non local maximal  LDIM  =      7   non local SUM 2l+1 LMDIM =     19
   total plane-waves  NPLWV = ******
   max r-space proj   IRMAX =      1   max aug-charges    IRDMAX=   7296
   dimension x,y,z NGX =    60 NGY =  180 NGZ =   96
   dimension x,y,z NGXF=   120 NGYF=  360 NGZF=  192
   support grid    NGXF=   120 NGYF=  360 NGZF=  192
   ions per type =              48  72

NPLWV is just the product of NGX, NGY and NGZ. So in your case NGXNGYNGZ=6018096.

I've looked into the code and NPLWV and NRPLWV are not the same. I've tested them (by manually writing out NRPLWV) and they can have very different values.
NRPLW is calculated as:
NRPLWV = 4 * 3.141592653589793238 * SQRT(ENCUT/13.605826)3 / 3 * OMEGA / 0.5291772493 / (2*3.141592653589793238)3 * 1.1 + 50

Here ENCUT is the plane wave cutoff and OMEGA is the volume of the cell (grep for "volume of cell" in the OUTCAR file).

Thank you for this answer!

I do have a follow up question regarding the inner-workings of VASP. On the webpage I linked earlier it mentions the estimates for memory used, but how is this value used within VASP? Does this give the maximum memory used throughout my entire calculation or does this give more an average? What flags within VASP and on clusters would be important to consider with this value in mind and how should it in general interact?

And is this the most important estimation that matters with regard to memory used or are there other processes which could overshadow this value by a large margin?

NRPLWV is the first dimension of the wave functions in allocation. So that will matter a lot.
Also important what I forgot to mention is that if you use the "gam" version of vasp, NRPLWV is halfed down. So if you can exploit that use it definitely.

I think the estimations on the memory requirements page make totally sense for standard VASP calculations. The part involving NRPLWV estimates the memory for wave functions and the second involving NGXF, NGYF and NGYZ estimates the charge density related stuff.
These will not give you exact numbers, because there is a lot of stuff like lapack-scalapack working arrays etc. that can also require some memory, but we assume that the wave functions and charge densities make out the majority of required memory.

Then my calculation might not be the standard calculation your are referring to as my numbers are entirely different from the estimates. What I see with the input files which are attached to this message is a maximum memory usage of about 2.3TB instead of about 50GB which I determined with the equation you gave me.

What is noteworthy is that the calculation is running for the entire duration and I can see that it converged in the output file, but it crashes afterwards due to this memory usage. What could be the cause of such a large difference in memory usage compared to the estimation? My current guess is that it has to do with writing out files after the calculation is done but I apart from making sure the WAVECAR file is not written out I'm unsure what else to change to reduce the memory load.

Your job is really large and I would not categorize it as a standard calculations (LORBIT etc. enabled). Hence manual memory estimates will be difficult.

Why it most likely runs out of memory after the SCF cycle is due to LORBIT calculations done after them, which require additional memory.

I have talked to colleagues and VASP should write out the memory usage and estimates on the first rank ("total amount of memory used by VASP MPI-rank0").
Could you have a look at that or upload the OUTCAR file?

You are using the line mode for K-POINTS with 4 segments. To reduce memory you could either split it in 4 separate calculations each with one segment or try the KPOINTS_OPT file. This way you can save 75 percent of memory.

I also wanted to adress ENCUT. You have a fairly high ENCUT of 800 eV. Have you optimized that value? Maybe you should start from a lower value and work your way up with convergence tests. In case you use a hard POTCAR like O_h you could switch to a POTCAR which requires a lesser ENCUT. These could further save you memory.

The value you mentioned in the OUTCAR file is 1170065. kBytes. I'm unsure if this value will be multiplied later and hence should be reduced or if it is just insignificant compared to the 2.3 TB VASP is requesting. So then I wonder, what does this value in the OUTCAR tell me?

I'll see if I can do 4 different calculations instead and change the ENCUT value. For now ENCUT has been chosen conveniently large so it should at least be able to be lowered a little.

The number is for a rank 0. So to get the total required memory you have to multiply with the number of ranks/cores you have. I see from the top of your OUTCAR file that you used 512 cores. So multiplying this number and dividing twice by 1024 (to convert from KB to GB) I get an estimate of 571 GB.

That's still much smaller than the 2.3 TB you saw. How did you get that number? By looking at the memory usage via top? Or is that the maximum memory by the nodes?

The 2.3 TB I got through the seff command after the job was out of memory. It states that it gives an estimate maximum for the memory (which seems to correspond with values I got from runs which finished.) Also when the program did crash it had a little over 1TB allocated in total which should be sufficient by the estimate you have just provided.

Your calculation seems to be correct so I'm not sure where the factor 4 difference could be coming from.

Also as an update, splitting the total band structure into segments works nicely. It reduces the maximum memory used by the expected amount.