Reproducing Goncharova et al. 2011: oxygen exchange and interface growth under a hafnium silicate film

One of the [studies we reproduced](/writing/interface-studies-reproduced) for [Questions we need Emily to answer](/writing/questions-for-emily-testing-a-theory-of-the-interface), the companion to [the essay](/writing/testing-a-theory-of-the-interface-with-an-ontology). Generated by `scripts/reproduce_studies.py` from `reproductions/goncharova_2011.py`. Every value from the paper was read by an AI agent and has not been checked by a person; the sample and history were re-read from the PDF alone by a second agent that had not seen our earlier transcription.

The study and why we depend on it

L. V. Goncharova, M. Dalponte, T. Feng, T. Gustafsson, E. Garfunkel, P. S. Lysaght and G. Bersuker, “Diffusion and interface growth in hafnium oxide and silicate ultrathin films on Si(001)”, Phys. Rev. B 83, 115329 (2011).

The challenge's reference for the hafnium stack. The graph holds 15 isotope amounts at five temperatures from the vector drawing of its Figure 5, and 9 depth-profile points from Figure 4(b). The exchange program has six fitted numbers; 1073 K and the depth profile were kept out of the fit.

How well can this study be reproduced?

Moderate. The anneal itself is well stated: gas, isotope enrichment, pressure, temperature and time. What came before it is not: the interlayer and its nitrogen, the time at temperature in vacuum, and whether each temperature is a fresh sample. The amounts are on a figure, and most were used in the fit.

The grade is ours and is about the paper as a source for simulation, not about the quality of the experiment: high means the sample, every step and its timing are stated well enough that a simulation needs no choices of ours; moderate that some choices are ours and are listed; low that the result depends on choices the paper does not constrain.

The sample and its history, played back

The sample is drawn from the simulation: layer heights and the read-outs are what the ISL programs compute at each moment of the history. Steps the language cannot state (a clean, a metallization, a measurement) are on the clock and marked as recorded, not simulated.

Does our sample and history match the paper's?

Left: the paper's own words, with the page of the PDF. Right: what we simulate. 5 same, 1 approximated, 3 assumed, 1 differs, 3 not in ours.

The timing of the history

Time is counted from the start of the first simulated step. A step the language cannot state takes no simulated time; where the paper gives its duration, the duration is in the last column. Stated means the paper gives the duration; partly stated that it gives a bound or a range; not stated; assumed that the duration in our history is ours.

763 K (490 °C), 30 min

Simulated time in this history: 30 min 01 s.

Across the samples

Timing the paper does not give

• How long the sample is held at temperature in vacuum before oxygen is admitted, and the ramp to that temperature.
• How fast the gas is pumped out and the sample cooled at the end.
• Whether the five temperatures are five samples or one sample annealed again and again. Figure 2's legend (“10 min”, “+30 min”) shows cumulative exposure was used at least once.
• The exposure range is “5–30 min” in the experimental section; the series we use is 30 min.
• The lowest temperature is 763 K in the figures, 490 °C in Table I and “~773 K” in the discussion.

What the second reading changed

• The stored recipe gives the anneal as dry oxygen at one atmosphere. The paper's gas is 18O2 at 0.01 Torr, about 1.3 × 10^-5 atm. No program reads the pressure, so the numbers do not change, but the fitted rates belong to that pressure and cannot be carried to another.
• Our oxygen dataset labels the lowest temperature 773 K and our profile dataset labels the same anneal 763 K. The paper uses both, and 490 °C; the figures say 763 K.
• The film sits on about 6 Å of oxynitride made with an ammonia anneal (973 K, 60 s). Our sample has no interlayer and no nitrogen; our interfacial oxide is growth only.
• Before the oxygen anneal the sample was heated briefly at 523 to 573 K to remove carbon and then stabilized at the anneal temperature in vacuum. Neither step is in our history.

Their terms and ours

The simulation in our language

Each sample is one history file: a SAMPLE and a CONTROL_TRAJECTORY. This is the whole input; run it with PYTHONPATH=. python -m isl_harness.predict <file>.

763 K (490 °C), 30 min

# 763 K (490 °C), 30 min
SAMPLE Film_763K {
    FAMILY: HighK_HfSiO_Si
    ORIENTATION: Si001
    FILM: Hf0.67Si0.33O2_27A
    INTERLAYER: SiOxNy_6A
}
CONTROL_TRAJECTORY History {
    EVENT Deposit KIND deposition TIMING assumed                 # Silicate film by ALD on a 1 nm interlayer
    EVENT Nitride KIND other TIMING stated                       # Ammonia anneal, 973 K, 60 s
    EVENT Degas KIND clean TIMING assumed                        # Brief anneal to remove surface carbon
    RUN Stabilize DURATION 1 TEMP 489.85->489.85 AMBIENT vacuum TIMING assumed VARY duration 1..1800
    RUN Anneal DURATION 1800 TEMP 489.85->489.85 AMBIENT mix:O2=1.316e-05 TIMING stated VARY duration 300..2400
    EVENT Cool KIND other TIMING assumed                         # Gas pumped out, sample cooled
    EVENT MEIS KIND measurement TIMING assumed                   # Ion scattering, in the same vacuum system
}

883 K (610 °C), 30 min

# 883 K (610 °C), 30 min
SAMPLE Film_883K {
    FAMILY: HighK_HfSiO_Si
    ORIENTATION: Si001
    FILM: Hf0.67Si0.33O2_27A
    INTERLAYER: SiOxNy_6A
}
CONTROL_TRAJECTORY History {
    EVENT Deposit KIND deposition TIMING assumed                 # Silicate film by ALD on a 1 nm interlayer
    EVENT Nitride KIND other TIMING stated                       # Ammonia anneal, 973 K, 60 s
    EVENT Degas KIND clean TIMING assumed                        # Brief anneal to remove surface carbon
    RUN Stabilize DURATION 1 TEMP 609.85->609.85 AMBIENT vacuum TIMING assumed VARY duration 1..1800
    RUN Anneal DURATION 1800 TEMP 609.85->609.85 AMBIENT mix:O2=1.316e-05 TIMING stated VARY duration 300..2400
    EVENT Cool KIND other TIMING assumed                         # Gas pumped out, sample cooled
    EVENT MEIS KIND measurement TIMING assumed                   # Ion scattering, in the same vacuum system
}

973 K (700 °C), 30 min

# 973 K (700 °C), 30 min
SAMPLE Film_973K {
    FAMILY: HighK_HfSiO_Si
    ORIENTATION: Si001
    FILM: Hf0.67Si0.33O2_27A
    INTERLAYER: SiOxNy_6A
}
CONTROL_TRAJECTORY History {
    EVENT Deposit KIND deposition TIMING assumed                 # Silicate film by ALD on a 1 nm interlayer
    EVENT Nitride KIND other TIMING stated                       # Ammonia anneal, 973 K, 60 s
    EVENT Degas KIND clean TIMING assumed                        # Brief anneal to remove surface carbon
    RUN Stabilize DURATION 1 TEMP 699.85->699.85 AMBIENT vacuum TIMING assumed VARY duration 1..1800
    RUN Anneal DURATION 1800 TEMP 699.85->699.85 AMBIENT mix:O2=1.316e-05 TIMING stated VARY duration 300..2400
    EVENT Cool KIND other TIMING assumed                         # Gas pumped out, sample cooled
    EVENT MEIS KIND measurement TIMING assumed                   # Ion scattering, in the same vacuum system
}

1073 K (800 °C), 30 min

# 1073 K (800 °C), 30 min
SAMPLE Film_1073K {
    FAMILY: HighK_HfSiO_Si
    ORIENTATION: Si001
    FILM: Hf0.67Si0.33O2_27A
    INTERLAYER: SiOxNy_6A
}
CONTROL_TRAJECTORY History {
    EVENT Deposit KIND deposition TIMING assumed                 # Silicate film by ALD on a 1 nm interlayer
    EVENT Nitride KIND other TIMING stated                       # Ammonia anneal, 973 K, 60 s
    EVENT Degas KIND clean TIMING assumed                        # Brief anneal to remove surface carbon
    RUN Stabilize DURATION 1 TEMP 799.85->799.85 AMBIENT vacuum TIMING assumed VARY duration 1..1800
    RUN Anneal DURATION 1800 TEMP 799.85->799.85 AMBIENT mix:O2=1.316e-05 TIMING stated VARY duration 300..2400
    EVENT Cool KIND other TIMING assumed                         # Gas pumped out, sample cooled
    EVENT MEIS KIND measurement TIMING assumed                   # Ion scattering, in the same vacuum system
}

1223 K (950 °C), 30 min

# 1223 K (950 °C), 30 min
SAMPLE Film_1223K {
    FAMILY: HighK_HfSiO_Si
    ORIENTATION: Si001
    FILM: Hf0.67Si0.33O2_27A
    INTERLAYER: SiOxNy_6A
}
CONTROL_TRAJECTORY History {
    EVENT Deposit KIND deposition TIMING assumed                 # Silicate film by ALD on a 1 nm interlayer
    EVENT Nitride KIND other TIMING stated                       # Ammonia anneal, 973 K, 60 s
    EVENT Degas KIND clean TIMING assumed                        # Brief anneal to remove surface carbon
    RUN Stabilize DURATION 1 TEMP 949.85->949.85 AMBIENT vacuum TIMING assumed VARY duration 1..1800
    RUN Anneal DURATION 1800 TEMP 949.85->949.85 AMBIENT mix:O2=1.316e-05 TIMING stated VARY duration 300..2400
    EVENT Cool KIND other TIMING assumed                         # Gas pumped out, sample cooled
    EVENT MEIS KIND measurement TIMING assumed                   # Ion scattering, in the same vacuum system
}

The programs the predictor ran on these histories: experiments/goncharova_oxygen/exchange_surface_limited.isl.template.

The model itself, experiments/goncharova_oxygen/exchange_surface_limited.isl.template, as it is written. Every equation the simulation uses is in this text; the runtime holds no physics.

# ==============================================================================
# ISL MODEL: OXYGEN EXCHANGE IN A HAFNIUM SILICATE FILM, LIMITED AT THE SURFACE
#
# The same program as exchange_transport.isl.template with one change: the film surface is
# not held at pure oxygen-18. Oxygen-18 enters at a finite rate, each atom replacing an
# oxygen-16 at the surface. With transport inside the film faster than that entry, the film
# is exchanged nearly uniformly instead of from the top down.
#
# The idea was proposed and screened in research/ontology_hypotheses (hypothesis H1) with one
# rate per temperature. Here the rate has a temperature law and is fitted, with the other
# rates, to the isotope amounts only; the depth profile and one temperature are left to test it.
# ==============================================================================
SAMPLE Film_{tag} {{
    FAMILY: HighK_HfSiO_Si
    ORIENTATION: Si001
}}
# Depth below the film surface. The film is 27 angstrom thick (paper, page 5).
GRID z {{ UNIT: cm; FROM: 0; TO: 2.7e-7; CELLS: 30 }}
FIELD C18 {{ UNIT: cm^-3; GRID: z; INITIAL: 0; MIN: 0; MAX: 1.0e23 }}
FIELD C16 {{ UNIT: cm^-3; GRID: z; INITIAL: C_tot; MIN: 0; MAX: 1.0e23 }}
# Oxygen that has left the film and been built into the interfacial oxide.
STATE N18_int {{ UNIT: cm^-2; INITIAL: 0; MIN: 0; MAX: 1.0e18 }}
STATE N16_int {{ UNIT: cm^-2; INITIAL: 0; MIN: 0; MAX: 1.0e18 }}
CONTROL T {{ UNIT: K }}
# Oxygen density of the film: the as-grown areal density read from Figure 5(a),
# 1.03e16 cm^-2, divided by the film thickness.
PARAMETER C_tot {{
    VALUE: 3.815e22
    UNIT: cm^-3
    STATUS: sourced
    SOURCE: Goncharova et al., PRB 83, 115329 (2011), Figure 5(a) as-grown oxygen content over 27 angstrom, agent-digitized
}}
# ---- fitted on the training temperatures -------------------------------------------
PARAMETER D0 {{ VALUE: 1.0e-12; UNIT: cm^2 s^-1; STATUS: fitted }}
PARAMETER E_D {{ VALUE: 0.5; UNIT: eV; STATUS: fitted }}
PARAMETER v0 {{ VALUE: 1.0e-8; UNIT: cm s^-1; STATUS: fitted }}
# Surface exchange: how fast oxygen of the gas replaces oxygen at the film surface. Finite, so the film can be partly exchanged all the way through.
PARAMETER s0 {{ VALUE: 1.0e-9; UNIT: cm s^-1; STATUS: fitted }}
PARAMETER E_s {{ VALUE: 0.5; UNIT: eV; STATUS: fitted }}
PARAMETER E_v {{ VALUE: 0.5; UNIT: eV; STATUS: fitted }}
PARAMETER k_B {{ VALUE: 8.617333262e-5; UNIT: eV K^-1; STATUS: constant }}
# The oxygen-16 still at the film surface: what an arriving oxygen-18 can replace. One in, one out.
QUANTITY surface_16 {{ UNIT: cm^-3; EQUATION: surface_16 = left(C16) }}
CONTROL_TRAJECTORY Anneal_{tag} {{
    RUN Anneal DURATION {anneal_s} TEMP {T_C}->{T_C} AMBIENT dry_O2
}}
# Three things are kept apart at the surface and the interface:
#   exchange   an oxygen-18 from the gas replaces an oxygen-16 at the surface (one in, one out; rate s0, E_s);
#   uptake     new oxygen enters from the gas, all of it oxygen-18, at the rate the drift carries oxygen away
#              from the surface, so the film's oxygen content stays what it was;
#   growth     oxygen leaving the bottom of the film is built into the interfacial oxide.
# Without the uptake term the film would lose to the interlayer oxygen that nothing replaced.
OPERATOR F_ExchangeTransport : Transport {{
    FLUX: J(C18) = -D0 * exp(-E_D / (k_B * T)) * grad(C18) + v0 * exp(-E_v / (k_B * T)) * C18
    FLUX: J(C16) = -D0 * exp(-E_D / (k_B * T)) * grad(C16) + v0 * exp(-E_v / (k_B * T)) * C16
    BOUNDARY: C18 AT left FLUX = s0 * exp(-E_s / (k_B * T)) * surface_16 + v0 * exp(-E_v / (k_B * T)) * C_tot
    BOUNDARY: C16 AT left FLUX = -s0 * exp(-E_s / (k_B * T)) * surface_16
}}
# Oxygen leaving the bottom of the film is built into the interfacial oxide, isotope by isotope.
OPERATOR F_InterfacialGrowth : OxideGrowth {{
    BOUNDARY: C18 AT right FLUX = v0 * exp(-E_v / (k_B * T)) * C18
    BOUNDARY: C16 AT right FLUX = v0 * exp(-E_v / (k_B * T)) * C16
    EQUATION: d(N18_int)/dt = v0 * exp(-E_v / (k_B * T)) * right(C18)
    EQUATION: d(N16_int)/dt = v0 * exp(-E_v / (k_B * T)) * right(C16)
}}
OBSERVATION_MODEL Obs_O18_film {{ TARGET: O18_film; UNIT: cm^-2; EQUATION: O18_film = integral(C18) }}
OBSERVATION_MODEL Obs_O18_interface {{ TARGET: O18_interface; UNIT: cm^-2; EQUATION: O18_interface = N18_int }}
OBSERVATION_MODEL Obs_O16_interface {{ TARGET: O16_interface; UNIT: cm^-2; EQUATION: O16_interface = N16_int }}
OBSERVATION_MODEL Obs_O_film {{ TARGET: O_film_total; UNIT: cm^-2; EQUATION: O_film_total = integral(C18) + integral(C16) }}
EVALUATION_CONTRACT Contract_{tag} {{
    FROZEN: true
    DATASET: data/derived/goncharova_2011_oxygen.csv
    DATASET_DIGEST: sha256:{dataset_sha256}
    SPLIT: {split}
{targets}
}}

Our simulation against the paper

16 within, 5 near, 3 off of 24 numbers. Tolerance is stated per row; nothing was fitted for this page.

Aligning the simulation: what the unstated things would have to be

Every step whose timing or temperature the paper leaves open carries a range in the history above (VARY). scripts/align_study.py searched those ranges against the paper's numbers through the same predictor, first one unknown at a time, then all together. No parameter a program declares fitted was touched. The unknowns were chosen on 1223K, 763K, 883K, 973K only; every other number is a test they played no part in.

Together: Stabilize duration 1 s (was 1); Anneal duration 1800 s (was 1800).

As reproduced: 16 of 24 within tolerance, median error 10.2%. Aligned: 16 of 24 within tolerance, median error 10.2%. On the held-out samples only: 0 of 3 within tolerance, median error 36.3% as reproduced, 0 of 3 within tolerance, median error 36.3% aligned.

Choices of program or material property, each scored on its own and on top of the aligned history:

What was searched. Two things the paper leaves open, as unknowns: the time the sample sits at temperature in vacuum before the gas is admitted (a new step, Stabilize, 1 s to 30 min; 1 s stands for none) and the anneal time (5 to 40 min: the text gives “5–30 min”, the series “30 min”, and Figure 2 uses 10 + 30 min). Two readings are variants: one film taken through every temperature in turn, and 10 min then 30 min more at each temperature. They were chosen on 763, 883, 973 and 1223 K (21 values); the three amounts at 1073 K are the test, as in the program's own fit. The starting interlayer could not be searched: the program reads no sample field for it and its interface reservoirs start at zero.

The anneal time is pinned at 30 min, but that is the fit, not the paper. Alone, 30 min gives 16 of 21 (median 6.1%), 28 min 16 of 21 (7.4%), 20 min 4 of 21 (25.8%, every value low), 40 min 9 of 21 (15.9%, high). The program's six numbers were fitted to these amounts with 30 min, so any other time is bound to do worse.

The vacuum hold is not a test of anything: the program does not read the gas. Every amount comes out the same with the anneal written in argon, in vacuum, in 18O2 at 0.01 Torr or in oxygen at one atmosphere (the check variant: 16 of 24, median 10.2%, identical to the reproduction). So a hold “in vacuum” is simulated as more time in 18O2, and the scan says only what the anneal-time scan says: up to about 40 s is harmless (16 of 21), 2.5 min loses one value (15), 8.6 min leaves 9 and 30 min leaves 5 (median 57%, every value high). Whether a 27 Å silicate grows interfacial oxide in vacuum before the gas, which the paper's crystallization anneal suggests it does, cannot be asked of this program.

The held-out 1073 K amounts do not move: 0 of 3 within tolerance before and after (median 36.3%). The interfacial 18O at 1073 K is 5.7 × 10^15 cm^-2 measured against 2.33 simulated, and the interfacial 16O 3.0 against 4.09: the paper's 1073 K interface already holds nearly as much 18O as at 1223 K (6.49), a step the program's single temperature law does not make.

The variants. One film taken through every temperature in turn gives 7 of 24 (median 33.8%, all high; the 1223 K interfacial 18O reaches 19.5 against 6.49), so the paper's Figure 5 is not one film re-annealed; the reading that each temperature is a fresh film stands. Ten minutes then thirty more gives 10 of 24 (median 19.8%); on 1073 K alone it brings the film's 18O within tolerance (7.51 against 7.38) and the median there to 33.7%, but it costs 7 of the 16 chosen values, which were fitted with 30 min.

1 near, 2 off of 3 numbers with the aligned settings (the samples used to choose the unknowns are left out).

The gas is the gap alignment cannot reach. The exchange program has no term in the oxygen pressure or the isotope: it adds 18O at a rate set by temperature alone, whatever the history says the sample is in. A vacuum hold, an argon anneal and an anneal in natural oxygen are all simulated as 18O2 at 0.01 Torr. A boundary flux proportional to p(18O2), and a reservoir of 16O that a vacuum anneal could move to the interface, are needed before the vacuum question can be asked.

The starting interlayer (about 6 Å of SiOxNy, about 2.6 × 10^15 oxygen atoms per cm² if it were SiO2, and its nitrogen) is not in the program's state, so the measured interfacial 16O (0.3 to 5.5 × 10^15) is compared with growth only. Whether the paper's interfacial 16O is likewise the growth above the as-deposited interlayer is a question for the paper, not for alignment.

What our language and ontology cannot yet say

• An isotope in a gas: mix:18O2=0.98,16O2=0.02 PRESSURE 0.01 Torr, so that the enrichment and the total pressure are stated and a program can read them. Today the isotope is implicit in the program and the pressure, though written, reaches nothing.
• A layer in the sample with its own composition that a program reads as an initial state (LAYER interlayer SiOxNy THICKNESS 6 Å O:N 5:1), so that the 6 Å oxynitride and its nitrogen start in the interface reservoir instead of being left out.
• A check in the predictor that a program's operators can act in the history's gas: a program whose source term needs oxygen should refuse, or warn, on a RUN in vacuum or argon instead of acting as if the gas were present.
• Measurement in the middle of a history (MEASURE … AFTER Anneal10min, then more steps): Figure 2's 10 min and +30 min are one film measured twice, which today must be two separate samples.
• Temperature in K as a unit on TEMP (TEMP 763 K): 763 K is written 489.85 °C, and the paper's own 763 K, 773 K and 490 °C for the same anneal are easier to keep apart when the unit is the paper's.

With heating and cooling transitions

No paper we hold states how its wafers heated up or cooled down. The project owner specified scenarios for one Vitkavage wafer (isl_harness/vitkavage_transitions.py); isl_harness/thermal_transitions.py applies the same numbers to every sample here: a heat-up in nitrogen before the first furnace step (a hot push of 1 to 5 min, or a cold-furnace ramp of 60 to 120 min), the owner's radiative cooling knots after the last furnace step or after the paper's own pull (fast, or every time multiplied by 5), and ramps around an anneal at 300 to 500 °C. The paper's own steps are kept as written. All transitions are assumptions, written in the history with TIMING assumed.

The paper's own conclusions

What we would like Emily to check

1. Is the 10.3 × 10^15 cm^-2 of 16O the film alone, or film plus interlayer?
2. Would a 27 Å silicate film on oxynitride change at 1223 K in vacuum, before any oxygen is admitted, enough to matter?
3. At 0.01 Torr, is exchange at the surface expected to be limited by arrival from the gas? If so the pressure belongs in the model.
4. Is each temperature in Figure 5 a separate sample?