MLFF NPT AIMD stops updating output after 3 ionic step

[dominic_varghese]

Hi everyone,

I am trying to set up active learning sampling in VASP (MLFF on-the-fly training) to generate a training dataset for MLIP development. I am running finite-temperature MD (NPT, Langevin thermostat) with ML_MODE = TRAIN. However, in my current setup, the run only prints output for the first 3 ionic steps, and after that no further output is written (OUTCAR, XDATCAR, etc.). I have let the job run for 24 hours and even 72 hours, but nothing new is printed beyond the 3rd step. The job does not crash or exit, and it continues to occupy resources, but appears to be “stuck” from an output point of view.

My goal is to generate a well-sampled ab initio training dataset via active learning that can later be used for MLIP training. I am attaching the relevant input files (INCAR, POSCAR, KPOINTS, POTCAR) for reference. Is there anything incorrect or missing in my INCAR settings for MLFF active learning?

MLFF.zip

Any guidance or suggestions would be greatly appreciated.

Thanks
Dominic

[ferenc_karsai]

Which toolchain are you using?

[dominic_varghese]

Hi Ferenc,

I am using the VASP version 6.5.0 and the below is the exact submit script I am using for the run.

module purge
module load intel/2018.release
module load mkl/2018.release
module load wannier/3.1.0
module load hdf5/hdf5-1.10.9-intel-2021-parallel

export HDF5_USE_FILE_LOCKING=FALSE

module load vasp/6.5.0

mpirun -np 16 vasp_std > log

[ferenc_karsai]

I don't have the same toolchain as you but I tried an intel+mkl and gnu+mkl one (with 32 and 16 cores). Both run fine, so I could not reproduce your hangup.

Can you please check if the hangup is reproducable with your current setup? If yes please check if it persists with another toolchain.

By the way you wrote that you ran with "mpirun -np 16 ...", but in the OUTCAR that you have uploaded 32 cores were used.

[dominic_varghese]

Sorry for the confusion. I had tried with both 16 and 32 cores in my current setup. The submit script command happened to be from the 16 core run while the OUTCAR was from the 32 core run. Both runs were giving me the halt at the same point (stopped running after 3 steps with no errors/logs).

I was able to run AIMD NPT run at 1000K in this current toolchain without any issue. I just checked with a intel-2021 version, that also ended updating in 3rd step. I can definitely re-compile my VASP with intel+mkl or gnu+mkl and see if I can complete the run and shall update here. Is there a recommended toolchain/module version for the MLFF part?

Thanks
Dominic

[ferenc_karsai]

Everything should work, so there is no real recommendations.

Maybe intel+mkl is the safest, because both are from the same company.

[dominic_varghese]

I have tested this calculation with multiple VASP versions (6.5.0 and 6.5.1) using different Intel + MKL compilation setups, but the issue persists. I also tried modifying the POSCAR, and with the current setup the calculation runs for only two ionic steps and then stops updating the log files.

At present, the job completes two steps in about five minutes, after which there are no further updates to the OUTCAR or log file(for 2 hours) - there is no error message / progress.

I am relatively new to running ML_MODE calculations, so I am not fully confident that my current INCAR settings are correct. It is also possible that I am making a mistake in one of the input files.

I am attaching the latest input and output files from this calculation for reference. I would greatly appreciate any guidance or suggestions on what might be going wrong.

6.5.1.zip

[dominic_varghese]

Hi Ferenc,

I tried with a different version of the intel+mkl toolchain and now the job runs and I can see the MLFF skipping configurations that are closely related to previous calculations.

Thanks for the support
Dominic

[ferenc_karsai]

Ah that's good to hear. Thanks for reporting back.

I've also looked at your input files, they look totally reasonable.

I will lock this topic.