ML LOGFILE - Revision history

Singraber at 09:41, 16 March 2026

2026-03-16T09:41:51Z

← Older revision		Revision as of 09:41, 16 March 2026
Line 394:		Line 394:

	=== <code>NORME</code> line ===		=== <code>NORME</code> line ===
	{{~~Not_released~~}}		{{available\|6.6.0}}
	The log lines starting with the <code>NORME</code> keyword contain a normalized version of the RMSEs given in the <code>ERR</code> line. The "normalizer" used here is the standard deviation computed from the training data. This quantity is sometimes abbreviated NRMSE<sub><math>\sigma</math></sub> or RSR (RMSE-observations standard deviation ratio). Values in the <code>NORME</code> line are already multiplied with 100%, i.e., expressed in percent.		The log lines starting with the <code>NORME</code> keyword contain a normalized version of the RMSEs given in the <code>ERR</code> line. The "normalizer" used here is the standard deviation computed from the training data. This quantity is sometimes abbreviated NRMSE<sub><math>\sigma</math></sub> or RSR (RMSE-observations standard deviation ratio). Values in the <code>NORME</code> line are already multiplied with 100%, i.e., expressed in percent.

Line 416:		Line 416:

	=== Per-species quantities for forces ===		=== Per-species quantities for forces ===
	{{~~Not_released~~}}		{{available\|6.6.0}}
	Some global quantities (global with respect to the training data set) given in the output lines of the {{FILE\|ML_LOGFILE}} are also available as per-atomic-species quantities, i.e., type-resolved results are provided in additional log lines:		Some global quantities (global with respect to the training data set) given in the output lines of the {{FILE\|ML_LOGFILE}} are also available as per-atomic-species quantities, i.e., type-resolved results are provided in additional log lines:

Singraber at 07:58, 24 October 2025

2025-10-24T07:58:03Z

Singraber: /* Collected data */

2025-10-16T21:24:30Z

Collected data

← Older revision		Revision as of 21:24, 16 October 2025
Line 261:		Line 261:

	== Collected data ==		== Collected data ==

	~~{{NB\|warning\|Work in progress...}}~~

	In this section we present additional in-depth information about the individual time series of data collected in the {{FILE\|ML_LOGFILE}}. As mentioned above, a short description is already provided directly in the file for each column of each keyword, e.g.		In this section we present additional in-depth information about the individual time series of data collected in the {{FILE\|ML_LOGFILE}}. As mentioned above, a short description is already provided directly in the file for each column of each keyword, e.g.

Singraber: /* Per-species quantities for forces */

2025-10-16T21:23:51Z

Per-species quantities for forces

← Older revision		Revision as of 21:23, 16 October 2025
Line 422:		Line 422:

	* <code>ERPS</code>: per-species RMSE of forces, corresponds to global force RMSE in <code>ERR</code>		* <code>ERPS</code>: per-species RMSE of forces, corresponds to global force RMSE in <code>ERR</code>
	* <code>NEPS</code>: per-species NRMSE<sub>σ</sub> of forces, corresponds to global force NRMSE<sub>σ</sub> in <code>NORME</code>		* <code>NEPS</code>: per-species NRMSE<sub><math>\sigma</math></sub> of forces, corresponds to global force NRMSE<sub><math>\sigma</math></sub> in <code>NORME</code>
	* <code>BEPS</code>,<code>BEFPS</code>: per-species Bayesian error estimate of forces, corresponds to global Bayesian error estimate of forces <code>BEE</code>,<code>BEEF</code>.		* <code>BEPS</code>,<code>BEFPS</code>: per-species Bayesian error estimate of forces, corresponds to global Bayesian error estimate of forces <code>BEE</code>,<code>BEEF</code>.
	* <code>SPFPS</code>,<code>SPFFPS</code>: per-species spilling factor of forces, corresponds to global spilling factor of forces <code>SF</code>,<code>SFF</code>.		* <code>SPFPS</code>,<code>SPFFPS</code>: per-species spilling factor of forces, corresponds to global spilling factor of forces <code>SF</code>,<code>SFF</code>.

Singraber: /* NORME line */

2025-10-16T21:23:02Z

NORME line

← Older revision		Revision as of 21:23, 16 October 2025
Line 397:		Line 397:
	=== <code>NORME</code> line ===		=== <code>NORME</code> line ===
	{{Not_released}}		{{Not_released}}
	The log lines starting with the <code>NORME</code> keyword contain a normalized version of the RMSEs given in the <code>ERR</code> line. The "normalizer" used here is the standard deviation computed from the training data. This quantity is sometimes abbreviated NRMSE<sub><math>\sigma</math></sub> or RSR (RMSE-observations standard deviation ratio).		The log lines starting with the <code>NORME</code> keyword contain a normalized version of the RMSEs given in the <code>ERR</code> line. The "normalizer" used here is the standard deviation computed from the training data. This quantity is sometimes abbreviated NRMSE<sub><math>\sigma</math></sub> or RSR (RMSE-observations standard deviation ratio). Values in the <code>NORME</code> line are already multiplied with 100%, i.e., expressed in percent.

	==== Energies ====		==== Energies ====

Singraber: /* Collected data */

2025-10-16T21:15:52Z

Collected data

@@ Line 341: / Line 341: @@
 \mathtt{STDAB}_S = \sqrt{\frac{1}{6M} \sum_{i=1}^{M} \sum_{\alpha\beta} \left(S^{\mathsf{dft}}_{\alpha\beta} - \overline{S^{\mathsf{dft}}_{\alpha\beta}}\right)^2} = \sqrt{ \frac{\sigma_{S,xx}^2 + \sigma_{S,yy}^2 + \sigma_{S,zz}^2 + \sigma_{S,xy}^2 + \sigma_{S,xz}^2 + \sigma_{S,yz}^2}{6} }
 </math>
 === <code>ERR</code> line ===

Singraber: /* Collected data */

2025-10-16T21:07:58Z

Collected data

@@ Line 342: / Line 342: @@
 </math>
 :<math>
 </math>
 ----
 [[Category:Files]][[Category:Machine-learned force fields]][[Category:Output files]]

Singraber at 13:44, 16 October 2025

2025-10-16T13:44:53Z

@@ Line 259: / Line 259: @@
   p 'err.dat' u 2:4 w l
 to show the force error along time steps.
 ----
 [[Category:Files]][[Category:Machine-learned force fields]][[Category:Output files]]

Schlipf at 13:35, 7 March 2025

2025-03-07T13:35:44Z

← Older revision		Revision as of 13:35, 7 March 2025
Line 261:		Line 261:
	----		----

	[[Category:Files]][[Category:Machine-learned force fields]][[Category:Output ~~Files~~]]		[[Category:Files]][[Category:Machine-learned force fields]][[Category:Output files]]

Singraber at 15:01, 19 October 2023

2023-10-19T15:01:14Z

← Older revision		Revision as of 15:01, 19 October 2023
Line 15:		Line 15:
	********************************************************************************************************************************************		********************************************************************************************************************************************

	The actual composition of log sections may depend on the machine learning mode of operation (see {{TAG\|~~ML_ISTART~~}}). Usually, in the beginning there will be a couple of sections describing the estimated memory consumption, machine learning settings and preexisting data. Then follows the main loop, which is split into a header and the actual loop body containing data describing the learning progress ({{~~TAG~~\|~~ML_ISTART~~}} ~~= 0, 1~~) or prediction ({{~~TAG~~\|~~ML_ISTART~~}} ~~= 2~~). Finally, there may be sections about actual memory consumption and timing statistics. The following chapters describe the contents of the log file sections in more detail:		The actual composition of log sections may depend on the machine learning mode of operation (see {{TAG\|ML_MODE}}). Usually, in the beginning there will be a couple of sections describing the estimated memory consumption, machine learning settings and preexisting data. Then follows the main loop, which is split into a header and the actual loop body containing data describing the learning progress ({{TAGO\|ML_MODE\|train}}) or prediction ({{TAGO\|ML_MODE\|run}}). Finally, there may be sections about actual memory consumption and timing statistics. The following chapters describe the contents of the log file sections in more detail:

	=== Memory consumption estimation ===		=== Memory consumption estimation ===

	This is usually the first section of the {{TAG\|ML_LOGFILE}} and contains an '''estimation''' of memory requirements based on {{VASP}} files read on startup. In the simplest case ({{~~TAG~~\|~~ML_ISTART~~}} ~~= 0~~) it depends on the settings in the {{FILE\|INCAR}} and {{FILE\|POSCAR}} file. For example, the expected memory consumption may vary with the number of elements present in the {{FILE\|POSCAR}} file. Various {{FILE\|INCAR}} tags also influence the memory demand, e.g. {{TAG\|ML_MB}} or {{TAG\|ML_MRB2}}. A continuation or prediction run, i.e. {{~~TAG~~\|~~ML_ISTART~~}} ~~= 1, 2~~, may also take settings from the files {{FILE\|ML_AB}} or {{FILE\|ML_FF}} into account.		This is usually the first section of the {{TAG\|ML_LOGFILE}} and contains an '''estimation''' of memory requirements based on {{VASP}} files read on startup. In the simplest case ({{TAGO\|ML_MODE\|train}}) it depends on the settings in the {{FILE\|INCAR}} and {{FILE\|POSCAR}} file. For example, the expected memory consumption may vary with the number of elements present in the {{FILE\|POSCAR}} file. Various {{FILE\|INCAR}} tags also influence the memory demand, e.g. {{TAG\|ML_MB}} or {{TAG\|ML_MRB2}}. A continuation or prediction run, i.e., {{TAGO\|ML_MODE\|train}} with {{FILE\|ML_AB}} present or {{TAGO\|ML_MODE\|run}}, may also take settings from the files {{FILE\|ML_AB}} or {{FILE\|ML_FF}} into account.

	* MEMORY INFORMATION ***********************************************************************************************************************		* MEMORY INFORMATION ***********************************************************************************************************************
Line 80:		Line 80:
	General settings		General settings
	--------------------------------------------------------------------------------------------------------------------------------------------		--------------------------------------------------------------------------------------------------------------------------------------------
	Machine learning operation mode : 0 (I) ~~ML_ISTART~~		Machine learning operation mode in strings (supertag) : REFIT (I) ML_MODE
			Machine learning operation mode : 4 ML_ISTART
			Precontraction of weights on Kernel for fast execution (ML_ISTART=2 only), but no error estimation : T ML_LFAST
			Controls the verbosity of the output at each MD step when machine learning is used : 1 ML_OUTPUT_MODE
			Sets the output frequency at various places for ML_ISTART=2 : 1 ML_OUTBLOCK


	Descriptor settings		Descriptor settings
	--------------------------------------------------------------------------------------------------------------------------------------------		--------------------------------------------------------------------------------------------------------------------------------------------
Line 101:		Line 105:
	=== Existing ab initio data ===		=== Existing ab initio data ===

	This section will appear in continuation runs (e.g. ~~<code>~~{{~~TAG~~\|~~ML_ISTART~~}} ~~= 1</code>~~) and summarizes the ab initio data found in the {{FILE\|ML_AB}} file.		This section will appear in continuation runs (e.g. {{TAGO\|ML_MODE\|train}} with existing {{FILE\|ML_AB}}) and summarizes the ab initio data found in the {{FILE\|ML_AB}} file.

	* AVAILABLE AB INITIO DATA *****************************************************************************************************************		* AVAILABLE AB INITIO DATA *****************************************************************************************************************
Line 115:		Line 119:
	=== Main loop ===		=== Main loop ===

	The central part of the {{FILE\|ML_LOGFILE}} is the main loop: depending on the machine learning mode {{TAG\|~~ML_ISTART~~}} it contains data collected over all the time steps (or other iterative schemes) along the {{VASP}} run. The main loop layout is carefully designed to minimize the file size while at the same being self-descriptive. To achieve this, it is split into two parts: the description blocks in the main loop header explain the available data and present its arrangement in lines and columns. Then, the main loop body contains the actual data (mostly raw numbers) in the previously defined layout. The separation of data and its description avoids unnecessary repetition and simplifies [[#Post-processing usage\|post-processing]].		The central part of the {{FILE\|ML_LOGFILE}} is the main loop: depending on the machine learning mode {{TAG\|ML_MODE}} it contains data collected over all the time steps (or other iterative schemes) along the {{VASP}} run. The main loop layout is carefully designed to minimize the file size while at the same being self-descriptive. To achieve this, it is split into two parts: the description blocks in the main loop header explain the available data and present its arrangement in lines and columns. Then, the main loop body contains the actual data (mostly raw numbers) in the previously defined layout. The separation of data and its description avoids unnecessary repetition and simplifies [[#Post-processing usage\|post-processing]].

	==== Header ====		==== Header ====

← Older revision		Revision as of 07:58, 24 October 2025
Line 15:		Line 15:
	********************************************************************************************************************************************		********************************************************************************************************************************************

	The actual composition of log sections may depend on the machine learning mode of operation (see {{TAG\|ML_MODE}}). Usually, in the beginning there will be a couple of sections describing the estimated memory consumption, machine learning settings and preexisting data. Then follows the main loop, which is split into a header and the actual loop body containing data describing the learning progress ({{~~TAGO~~\|ML_MODE\|train}}) or prediction ({{~~TAGO~~\|ML_MODE\|run}}). Finally, there may be sections about actual memory consumption and timing statistics. The following chapters describe the contents of the log file sections in more detail:		The actual composition of log sections may depend on the machine learning mode of operation (see {{TAG\|ML_MODE}}). Usually, in the beginning there will be a couple of sections describing the estimated memory consumption, machine learning settings and preexisting data. Then follows the main loop, which is split into a header and the actual loop body containing data describing the learning progress ({{TAG\|ML_MODE\|train}}) or prediction ({{TAG\|ML_MODE\|run}}). Finally, there may be sections about actual memory consumption and timing statistics. The following chapters describe the contents of the log file sections in more detail:

	=== Memory consumption estimation ===		=== Memory consumption estimation ===

	This is usually the first section of the {{TAG\|ML_LOGFILE}} and contains an '''estimation''' of memory requirements based on {{VASP}} files read on startup. In the simplest case ({{~~TAGO~~\|ML_MODE\|train}}) it depends on the settings in the {{FILE\|INCAR}} and {{FILE\|POSCAR}} file. For example, the expected memory consumption may vary with the number of elements present in the {{FILE\|POSCAR}} file. Various {{FILE\|INCAR}} tags also influence the memory demand, e.g. {{TAG\|ML_MB}} or {{TAG\|ML_MRB2}}. A continuation or prediction run, i.e., {{~~TAGO~~\|ML_MODE\|train}} with {{FILE\|ML_AB}} present or {{~~TAGO~~\|ML_MODE\|run}}, may also take settings from the files {{FILE\|ML_AB}} or {{FILE\|ML_FF}} into account.		This is usually the first section of the {{TAG\|ML_LOGFILE}} and contains an '''estimation''' of memory requirements based on {{VASP}} files read on startup. In the simplest case ({{TAG\|ML_MODE\|train}}) it depends on the settings in the {{FILE\|INCAR}} and {{FILE\|POSCAR}} file. For example, the expected memory consumption may vary with the number of elements present in the {{FILE\|POSCAR}} file. Various {{FILE\|INCAR}} tags also influence the memory demand, e.g. {{TAG\|ML_MB}} or {{TAG\|ML_MRB2}}. A continuation or prediction run, i.e., {{TAG\|ML_MODE\|train}} with {{FILE\|ML_AB}} present or {{TAG\|ML_MODE\|run}}, may also take settings from the files {{FILE\|ML_AB}} or {{FILE\|ML_FF}} into account.

	* MEMORY INFORMATION ***********************************************************************************************************************		* MEMORY INFORMATION ***********************************************************************************************************************
Line 105:		Line 105:
	=== Existing ab initio data ===		=== Existing ab initio data ===

	This section will appear in continuation runs (e.g. {{~~TAGO~~\|ML_MODE\|train}} with existing {{FILE\|ML_AB}}) and summarizes the ab initio data found in the {{FILE\|ML_AB}} file.		This section will appear in continuation runs (e.g. {{TAG\|ML_MODE\|train}} with existing {{FILE\|ML_AB}}) and summarizes the ab initio data found in the {{FILE\|ML_AB}} file.

	* AVAILABLE AB INITIO DATA *****************************************************************************************************************		* AVAILABLE AB INITIO DATA *****************************************************************************************************************
Line 342:		Line 342:
	==== Subset standard deviation ====		==== Subset standard deviation ====

	Calculation of mean values <math>\overline{e^{\mathsf{dft}}}</math>, <math>\overline{F^{\mathsf{dft}}_{\alpha}}</math>, <math>\overline{S^{\mathsf{dft}}_{\alpha\beta}}</math> and respective standard deviations <math>\sigma_{E}</math>, <math>\sigma_{F,\alpha}</math>, <math>\sigma_{S,\alpha\beta}</math> are modified for {{~~TAGO~~\|ML_IWEIGHT\|3}} to account for potentially disconnected regions in the training data. For example, consider a training data set which contains structures from ab initio MD simulations of two different crystal structures (e.g. fcc and bcc). The corresponding potential energies may then be clustered around two mean values separated by an energy range not represented in the data set. Computing <math>\sigma_{E}</math> directly from all energy data would then result in an overestimation of the desired measure for the data spread. Consequently, also the normalized root mean square error (<math>\mathtt{NORME}_E</math>) would result in unreasonably low values. It seems natural to split the calculation of the standard deviation along the clusters, compute separate values for each subset and finally average the standard deviations over both sets. In general, we adopt the following strategy for computing standard deviations for heterogeneous data sets: First, the training data is split into multiple subsets based on atom types, number of atoms per type and system names. For details about data set separation, see {{TAG\|ML_IWEIGHT}} and {{TAG\|ML_LUSE_NAMES}}. Let us assume we now have <math>R</math> subsets, where set <math>r</math> contains <math>M_r</math> structures, i.e.,		Calculation of mean values <math>\overline{e^{\mathsf{dft}}}</math>, <math>\overline{F^{\mathsf{dft}}_{\alpha}}</math>, <math>\overline{S^{\mathsf{dft}}_{\alpha\beta}}</math> and respective standard deviations <math>\sigma_{E}</math>, <math>\sigma_{F,\alpha}</math>, <math>\sigma_{S,\alpha\beta}</math> are modified for {{TAG\|ML_IWEIGHT\|3}} to account for potentially disconnected regions in the training data. For example, consider a training data set which contains structures from ab initio MD simulations of two different crystal structures (e.g. fcc and bcc). The corresponding potential energies may then be clustered around two mean values separated by an energy range not represented in the data set. Computing <math>\sigma_{E}</math> directly from all energy data would then result in an overestimation of the desired measure for the data spread. Consequently, also the normalized root mean square error (<math>\mathtt{NORME}_E</math>) would result in unreasonably low values. It seems natural to split the calculation of the standard deviation along the clusters, compute separate values for each subset and finally average the standard deviations over both sets. In general, we adopt the following strategy for computing standard deviations for heterogeneous data sets: First, the training data is split into multiple subsets based on atom types, number of atoms per type and system names. For details about data set separation, see {{TAG\|ML_IWEIGHT}} and {{TAG\|ML_LUSE_NAMES}}. Let us assume we now have <math>R</math> subsets, where set <math>r</math> contains <math>M_r</math> structures, i.e.,

	:<math>		:<math>