PBE0 hybrid convergence

[william_watson]

I am trying to calculate the ACFDT-RPA total energy for UN with 1k AFM magnetism. I am applying this across seven different lattice parameters with the intention of using the equation of state method to find the equilibrium lattice constant (between 0.97 and 1.03a where a = lattice constant) with the intention of comparing the E-V curves for PBE and PBE-RPA. For step 2 of the process, the 1.02a and 1.03a calculations do not converge across the maximum walltime (48h). I have steadily increased NBANDS over time to increase the number of empty bands to help achieve convergence but this has not helped. I've attached files from the 1.03a calculation which has reached maximum walltime. Calculations for 0.97/0.98/0.99/1/1.01a have all converged successfully with default number of NBANDS.

[william_watson]

I am not able to successfully run a PBE0 calculation for UN (using either v5.4.4 or 6.5.0). I start the calculation from a CHGCAR and WAVECAR obtained from a PBE calculation and have increased NBANDS and kpoints slightly relative to the PBE calculation. The calculation seems to get stuck at the first electronic step. I experience similar issues with other hybrids too. I've attached the input files.

[alexey.tal]

Dear william_watson,

Both calculations seem to hang in the same initial step, so I decided to merge these two topics, as they are likely related.

I suggest that we first try to address the issue in the PBE0 calculation. I do not see anything suspicious in the setup, and it also appears that the issue is present regardless of the VASP version.

Were you able to run this calculation with fewer k-points? Does the same problem occur in your calculation with the PBE functional?

Best regards,
Alexey

[william_watson]

Dear Alexey,

I did not encounter any issues with the PBE calculation. The PBE0 calculation does run if I do not initialise with PBE WAVECAR and with a reduced k-point mesh of 2x2x2.

Kind regards,

William

[alexey.tal]

Dear William,

Thank you for the clarification. I think that the issue here might the following. When you restart the calculation form the PBE wavefunctions the first step is going to involve the calculation of the exact exchange, which takes the majority of time and in fact this calculation is not hanging but rather taking a really long time to complete. For the case where you start your PBE0 calculation "from scratch" the first few iterations are essentially just the PBE ones.

The reason this calculation might take so long is that you disable the symmetry ISYM=0. I've tried running this calculation but we don't have a lot of resources available at the moment to run such a large test. I would suggest that you first try to make sure that your calculation need is hanging and not just taking a long time. The way to do it would be to gradually increase the k-points gird and see how the computational time increases. This should allow one to make en estimate of how long your final calculation for 14x14x14 might take. Also, I would suggest that you use the symmetry to speed up the calculation. Alternatively, you can use NKRED flag to reduce the k-points
sampling in the Fock term.

[william_watson]

Thank you Alexey. Would you also suggest this approach for the ACFDT calculation step (where ALGO = EIGENVAL) or are there any additional points which must be considered for this particular convergence problem?

Kind regards,

William

[alexey.tal]

Yes, I would suggest using symmetry and increasing k-points gradually in the case of ACFDT.

[william_watson]

Thank you Alexey - this approach has worked in step 2 of my ACFDT calculations and I am iteratively increasing k-point density with my hybrid calculations.

I have a final question regarding an error message generated in step 4 of my ACFDT. My error log contains several repeats of the following message:

internal error in KPOINT_IN_FULL_GRID: can not find 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000

Is this anything to be concerned about?

Thanks,

William

[alexey.tal]

Dear William,

Could you please provide the input and output files for this calculation?

I've seen this type of error when a shifted grid is used, while the symmetry is enabled. A way to avoid it would be setting ISYM=-1.

Best wishes,
Alexey

[william_watson]

Good morning Alexey,

Input and output files provided in zip file attached (apart from CHGCAR and WAVECAR generated in step 3 which were used to initialise this calculation).

Thanks,

Will

[alexey.tal]

I see that indeed you are using a Monkhorst-Pack grid, which does not include the gamma-point. You should use a gamma-centered grid instead.

But there is another issue that I see in your output files.

Code: Select all

 -----------------------------------------------------------------------------
|                                                                             |
|     EEEEEEE  RRRRRR   RRRRRR   OOOOOOO  RRRRRR      ###     ###     ###     |
|     E        R     R  R     R  O     O  R     R     ###     ###     ###     |
|     E        R     R  R     R  O     O  R     R     ###     ###     ###     |
|     EEEEE    RRRRRR   RRRRRR   O     O  RRRRRR       #       #       #      |
|     E        R   R    R   R    O     O  R   R                               |
|     E        R    R   R    R   O     O  R    R      ###     ###     ###     |
|     EEEEEEE  R     R  R     R  OOOOOOO  R     R     ###     ###     ###     |
|                                                                             |
|      The derivative of the wavefunctions with respect to k (WAVEDER)        |
|      can not be found. You should redo the groundstate calculations         |
|      using LOPTICS=.TRUE. in order to write the WAVEDER file.               |
|      For metals the present setting is however ok.                          |
|                                                                             |
 -----------------------------------------------------------------------------

Code: Select all

 -----------------------------------------------------------------------------
|                                                                             |
|           W    W    AA    RRRRR   N    N  II  N    N   GGGG   !!!           |
|           W    W   A  A   R    R  NN   N  II  NN   N  G    G  !!!           |
|           W    W  A    A  R    R  N N  N  II  N N  N  G       !!!           |
|           W WW W  AAAAAA  RRRRR   N  N N  II  N  N N  G  GGG   !            |
|           WW  WW  A    A  R   R   N   NN  II  N   NN  G    G                |
|           W    W  A    A  R    R  N    N  II  N    N   GGGG   !!!           |
|                                                                             |
|      The number of bands has been changed from the values supplied          |
|      in the INCAR file. This is a result of running the parallel version.   |
|      The orbitals not found in the WAVECAR file will be initialized with    |
|      random numbers, which is usually adequate. For correlated              |
|      calculations, however, you should redo the groundstate calculation.    |
|      I found NBANDS    =     2701  now  NBANDS  =    2752                   |
|                                                                             |
 -----------------------------------------------------------------------------

This error indicates that the number of bands changed between the DFT and ACFDT steps, which will make the WAVEDER file incompatible. So you should redo the DFT and ACDFT steps and make sure that the number of bands has not changed.

[william_watson]

Hi Alexey,

Thank you for your suggestions. I have restarted the calculation to ensure NBANDS remains consistent. I am now getting another error (seen in the UN_1k_GGA_ACFDT.sh.o* file in the zip attachment). Given the termination of the process, is this a memory issue?

Thanks,

Will

[alexey.tal]

Hi Will,

Yes, in your UN_1k_GGA_ACFDT.sh.e441134 you can see the line:

Code: Select all

Cgroup mem limit exceeded: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=441134.pbs-6,mems_allowed=0-1,oom_memcg=/pbs_jobs.service/jobid/441134.pbs-6,task_memcg=/pbs_jobs.service/jobid/441134.pbs-6,task=vasp_std,pid=229812,uid=1613728

This means that the process was killed by the operating system due to out of memory error.

This block from your OUTCAR suggests that the number of bands is not the same:

Code: Select all

found WAVECAR, reading the header
  number of bands has changed, file:  3244 present:  3264
  trying to continue reading WAVECAR, but it might fail
 POSCAR, INCAR and KPOINTS ok, starting setup
 FFT: planning ...
 reading WAVECAR
 VASP.5.3 WAVECAR encountered
 random initialization beyond band         3245
 the WAVECAR file was read successfully
 initial charge from wavefunction

 The Fermi energy was updated, please check that it is located mid-gap
 values below the HOMO (VB) or above the LUMO (CB) will cause erroneous energies
 E-fermi :  11.0065


 -----------------------------------------------------------------------------
|                                                                             |
|     EEEEEEE  RRRRRR   RRRRRR   OOOOOOO  RRRRRR      ###     ###     ###     |
|     E        R     R  R     R  O     O  R     R     ###     ###     ###     |
|     E        R     R  R     R  O     O  R     R     ###     ###     ###     |
|     EEEEE    RRRRRR   RRRRRR   O     O  RRRRRR       #       #       #      |
|     E        R   R    R   R    O     O  R   R                               |
|     E        R    R   R    R   O     O  R    R      ###     ###     ###     |
|     EEEEEEE  R     R  R     R  OOOOOOO  R     R     ###     ###     ###     |
|                                                                             |
|      The derivative of the wavefunctions with respect to k (WAVEDER)        |
|      can not be found. You should redo the groundstate calculations         |
|      using LOPTICS=.TRUE. in order to write the WAVEDER file.               |
|      For metals the present setting is however ok.                          |
|                                                                             |
 -----------------------------------------------------------------------------

[william_watson]

Apologies - I shared an example of one of the lattice parameter points which I had not updated. I have now attached copies of files where NBANDs remains consistent.

I killed the step 4 (ALGO = ACFDT) calculation as it had only worked over the first q-point after 20 hours. I think I can reduce k-point mesh size and use k-point parallelisation over the two nodes that I am allocating but even then it feels like these calculations may not complete within my maximum walltime (48h). Is there anything else in my INCAR file that you can see which would make the calculation more efficient?

I also have a couple of questions:

• My intention was to run this algorithm over several possible magnetic configurations for the structure (e.g. 1k AFM, 3k AFM transverse) therefore should I not turn symmetry off for these calculations?

• Should I consider the low-scaling method using ALGO = ACFDTR to improve efficiency

[alexey.tal]

Now I see that the number of bands is consistent between the calculations.

Should I consider the low-scaling method using ALGO = ACFDTR to improve efficiency?

The low-scaling algorithm is available starting from VASP 6.0, whereas you are currently using VASP 5.4.4. If you have access to a newer VASP version, you should definitely use it instead of the older one. However, note that the low-scaling algorithm is more memory demanding. You can find guidance on how to estimate the memory requirements for the low-scaling algorithm on our wiki.

My intention was to run this algorithm over several possible magnetic configurations for the structure (e.g. 1k AFM, 3k AFM transverse). Therefore, should I not turn symmetry off for these calculations?

I am not sure I fully understand why symmetry would need to be disabled in this case. Could you elaborate on your reasoning? As a practical check, you could also test your system with and without symmetry enabled and compare the energies at the DFT level to verify that the same solution is obtained.