error in MLFF

[yingma]

Dear admins,

I'm trying to develop a MLFF for amorphous Si. I performed on-the-fly training, followed by a refit calculation to generate MLFF. The number of configurations is 1629, and number of basis sets is 6784. From ML_LOGFILE for the refit calculation, the error in energy is 10.3 meV/atom and error in force is 0.15 eV/Angstrom.

Using the generated MLFF, I run an MD of amorphous Si. From the trajectory of the simulation, I selected 50 configurations. For each configuration, I performed single point DFT and MLFF calculations, based on which the mean absolute error and max error were calculated. The mean absolute error in energy is 10.6 meV/atom and in force is 0.14 eV/Angstrom, which seems to be consistent with those from the refit calculation. However, the max error in energy is 28.6 meV/atom, and 0.92 eV/Angstrom in force.

So my questions are:
1) The max error in energy and in force are much larger than the mean. For sure max error will be larger, but in my case the max error in force is almost 7 times larger than the mean - is this behavior normal? Can I trust the result from the MLFF? If not, any suggestions on how to minimize the max error?
2) VASP manual recommend an energy error of less than 1 meV/atom and force error of less than 0.1 eV/Angstrom. The errors I have are beyond those recommended values. Since I'm modeling amorphous materials, I guess a larger error is expected. What are typically errors for amorphous materials? Do you have any recommendation on how to minimize the training error?
3) I have 1629 configurations and 6784 local references. Are those numbers reasonable for amorphous materials?

Any insights will be greatly appreciated. Thank you very much.

[christopher_sheldon1]

Dear Yingma,

Thank you for your question. That is quite a large value. The max error being 7x larger isn't necessarily an issue. The mean error is more important. Could you upload your OUTCAR, POSCAR, KPOINTS, INCAR, and stdout files so that I can look into this further?

I am not sure about the number of configurations. It is a rather large number but I would have to look at your input and output files to answer more precisely.

Best wishes,

Chris

[yingma]

Hi Chris,

Thank you very much for your response. Please find all files using this https://universityofwieauclaire-my.shar ... M?e=IZ5FMM

Here are more details on my calculations: I used the melt-quench method to generate amorphous Si. All melt-quench simulations were performed using AIMD. Six 300K amorphous Si structures are selected for MLFF training (POSCAR.1 to POSCAR.6). For each POSCAR, three scaling factors, 0.95, 1.0, and 1.05 were used, and thus there are totaling 15 structures used for on-the-fly training. Each training was performed at 500K for 2 ps. ML_ABN and ML_LOGFILE from each training are saved in folder savedfile.

After training, the ML_ABN from the last training was used for refit calculation. The ML_FF file from refit is then used to run MD simulations for each of the POSCAR. The errors I mentioned in my previous post is from POSCAR.3, but other POSCARs show similar behavior. The summary of errors is in file "applyerror.dat".

Maybe the large error is related to the way I train the MLFF. Any recommendations on how to properly train MLFF for amorphous materials would also be greatly appreciated.

Thank you very much!

[christopher_sheldon1]

Dear Yingma,

The reason for the large mean error in energy and forces is that the dataset is very broad, so the MLFF struggles to fit it. Hence, if you take a set of configurations from its training set, it cannot reproduce them. There are plenty of configurations (1600-1700), but the dataset itself seems very broad. Do you have the INCAR file that you used to train the MLFF? Then I can make some suggestions as to what to improve. Are you trying to train liquid Si? The temperature range will be important.

In answer to your original questions:
1) Max error can be larger - not necessarily an issue. Mean aboslute error is more important usually.
2) Amorphous materials may well have larger errors but not so much larger. If they are, then the MLFF is not fitting well.
3) Lots of configurations and local references, shouldn't be the issue.

Best wishes,

Chris

[yingma]

Thank you, Chris. Don't we want to train MLFF using a broad dataset? Especially for amorphous materials, the structures can be diverse with a lot of different local configurations.

Here is the INCAR

SYSTEM = Si Train 1
LANGEVIN_GAMMA = 50.0
TEBEG = 300
TEEND = 500
ML_IALGO_LINREG = 1
ML_SION1 = 0.3
ML_MRB2 = 12
NSW = 5000
####
PREC = Normal
NELMIN = 5
NELMDL = 5
EDIFF = 1.0e-05
ALGO = Normal
ISTART = 0
ISMEAR = 0
SIGMA = 0.1
NPAR = 8
LREAL=A
ISYM=0
#MD
NSW=5000
NBLOCK=500
IBRION=0
IWAVPR=11
POTIM=1.0
MDALGO=3
LCHARG=.FALSE.
NWRITE=0
ISIF=2
ML_LMLFF = .TRUE.
ML_MODE = TRAIN
#ADDED MLFF PARAMETERS
ML_MCONF=3000
ML_MB=8000
#####

[christopher_sheldon1]

Hi Yingma,

Yes, you want to have a broad dataset. It's important to not make it so broad that it can't be fitted though. You've got narrow volume ranges as far as I can tell. It might help to set ISIF=3 for your system; then you'd have a little bit of fluctuation in the volume while keeping the cell the same shape (https://vasp.at/wiki/ICONST#Restriction ... ation_cell). This should improve things. Your setting of NSW=5000 seems quite short for an MD simulation, so it might be worth increasing so that you can sample a wider range of structures.

EDIT:
I've just discussed this with one of the MLFF developers who's tried liquid Si, and he found similar issues when training an MLFF, getting a maximum force of 0.7 eV/Angstrom in force. This indicates that this is a difficult system to model. He used a few hundred ps (so NSW = 100000 or more) but try NSW = 10000 and see, which will make the MLFF more stable; it may increase the stability of the MLFF but make it less accurate. You could try to increasing the temperature range to but this will increase the spread and make it harder to fit. Check the structure throughout the MD and see if these look reasonable. If you could upload the conditions for the on-the-fly (ML_MODE=run) input settings we can take a look at these and try to make some more suggestions. It is probably a limitation of the system for any force field to fit (liquid Si is a known problem system, and your system is similar to it).

Best wishes,

Chris

[yingma]

Hi Chris,

Thank you very much for your answers. I trained the MLFF in NVT based on recommendations from MedeA developers. They mentioned "basis set reinitialization" problems and recommended to train MLFF under NVT using using a few different scaling factors. I'll certainly try using NPT and see if things improve.

Liquid (or amorphous) Si may be a problematic system - but do you have any insights as to why it is problematic?

Below is my INCAR for ML_MODE=RUN:

SYSTEM = Si equilibrium (from 6008)

PREC = Normal
ENCUT = 245.345
IBRION = 0
NSW = 30000
ISIF = 3
NELMIN = 2
EDIFF = 1.0e-05
EDIFFG = -0.02
VOSKOWN = 1
ISYM = 0
POTIM = 1
NBLOCK = 100
TEBEG = 300
TEEND = 300
NWRITE = -1
MDALGO = 3
LANGEVIN_GAMMA = 50.0
LANGEVIN_GAMMA_L = 10.0
ISYM = 0
NELM = 60
ALGO = Fast (Davidson and RMM-DIIS)
ISPIN = 1
INIWAV = 1
ISTART = 0
ICHARG = 2
LWAVE = .FALSE.
LCHARG = .FALSE.
ADDGRID = .FALSE.
ISMEAR = 0
SIGMA = 0.05
LREAL = Auto
LSCALAPACK = .TRUE.
RWIGS = 1.11
#PMASS = 100
#PSTRESS = 0.001
#MLFF
ML_LMLFF = .TRUE.
ML_MODE = RUN
ML_OUTBLOCK=100
ML_ESTBLOCK=100

[christopher_sheldon1]

Hi Yingma,

I'm afraid that we don't know why liquid/ amorphous Si is problematic. The solid behaves well, and many liquids behave well, so it is something to do with the specifics of the system that we are not aware of.

I'd recommend increasing TEEND to at least 500, perhaps 700. Having only a very narrow temperature range will make the MLFF unreliable for temperatures outside of this range. Increasing the ENCUT to be around 30 % more than the ENMAX in the POTCAR might also help, giving slightly more converged energies and forces.

Suggestions:

• Increase TEEND to 500 K

• Increase ENCUT to 320 eV

• Run for longer - increase NSW to 100000

• Put ML_SION1 = 0.3 and ML_MRB2 = 12 in the refit INCAR also

• Try NpT and only explore volumes around the starting value

Besides these that I've given before, I don't have any other suggestions. I hope that this helps.

Best wishes,

Chris

[yingma]

Thank you, Chris. I appreciate your help. I'll try some NPT runs.