MLFF RMSE question

Question on input files/tags, interpreting output, etc.

Please check whether the answer to your question is given in the VASP online manual or has been discussed in this forum previously!

Moderators: Global Moderator, Moderator

Locked
Message
Author
xiliang_lian
Newbie
Newbie
Posts: 37
Joined: Thu May 19, 2022 8:44 am

MLFF RMSE question

#1 Post by xiliang_lian » Thu Sep 08, 2022 2:48 pm

Hello,

I have started to use the machine learning method implemented in VASP 6.3.0 and I have a question about RMSE implemented here. I tried to find solutions on the forum, wiki, and some initial publications by Georg Kresse, but I didn't find a clear answer.

Could you please explain how the RMSE is calculated in VASP? If I understand correctly, it is a cumulative error, right? If it is, we can observe a decrease in RMSE, is this because the RMSE has been averaged over all the configurations in the datasets? If not, from both the example of melting Si provided by you and the calculation I made, I always observe a steady increase. This clearly indicates the model is performing worse and doesn't make sense. Can you please clarify this point a little bit?

Thanks a lot in advance.

Best regards,
Xiliang

ferenc_karsai
Global Moderator
Global Moderator
Posts: 422
Joined: Mon Nov 04, 2019 12:44 pm

Re: MLFF RMSE question

#2 Post by ferenc_karsai » Mon Sep 12, 2022 8:57 am

The RMSE is calculated after an update of the force field over all training structures up to that step. So yes the RMSE can go up and down.
I have clarified the section for "ERR" in the monitoring part of the best practices for machine-learned force fields:
https://www.vasp.at/wiki/index.php/Best ... Monitoring

So if RMSE goes up what can cause that:
1) First check if you have a temperature ramp. If yes then increased temperature always leads to larger errors due to a broader variety of configurations.
2) If the RMSE suddenly goes up it could also mean that you encounter some kind of deformation of your cell. In that case your force field is not good enough to describe those structures. This is usually something you want to avoid if you want to train a specific phase. So in that case adjust your MD parameters. For melting for example this definitely happens. But there one would possibly use the force field with on-the-fly learning to get a melt from the crystal structure, but throw away the force field created during the melting process. Afterwards one would train on the melt and crystal structure separately.
3) The evidence approximation is unable to fit (can sometimes happen for magnetic structures). In this case some investigations is needed and some parameters for the algorithm need to be adjusted. But this is the least likely to happen since Tikhonov regularization is turned on by default.

Please check if you encounter 1) or 2).

xiliang_lian
Newbie
Newbie
Posts: 37
Joined: Thu May 19, 2022 8:44 am

Re: MLFF RMSE question

#3 Post by xiliang_lian » Tue Sep 13, 2022 1:00 pm

Thank you very much. It is very clear to me.

Locked