AssertError and improve speed when using simulate_waveforms()

[zql2021]

Hello,sorry to bother you again. I've encountered some issues while using the simulate_waveforms(),hope you can give me some advice.

1. I received an AssertionError stating that the center of the charge cloud (Float32[0.013906903, -0.0041047907, 0.00090030907]) is not inside the semiconductor. However, the coordinate [0.013906903, -0.0041047907, 0.00090030907] is indeed within my detector, which makes me puzzled.Is there something wrong with my detector config? The file is attached at the end.

using Suppressor
@suppress_err contact_charge_signals = simulate_waveforms(      
        filtered_events[1:1000],
        sim,
        max_nsteps = 4000, 
        Δt = 1u"ns", 
        number_of_carriers = 100,
        diffusion = true, 
        self_repulsion = true, 
        verbose=true);

2. I want to save waveforms,so I use

import HDF5
using Suppressor
@suppress_err simulate_waveforms(      
    filtered_events[1:1000],
    sim,
    "ssd/outdata",
    "waveform",
    max_nsteps = 4000, 
    Δt = 1u"ns", 
    number_of_carriers = 100, 
    diffusion = true, 
    self_repulsion = true, 
    verbose=true);

but received an ArgumentError. If I delete output_dir and output_base_name,it can work.

3.Performance of simulate_waveforms().simulate_waveforms() is running slowly, and it seems to use only single thread. When I run simulate!(), it used 20 threads. So I'm curious if simulate_waveforms() can support multi thread?

4. Suppressing warnings and seeing the progress bar: I've used @suppress_err to suppress warning outputs, which unfortunately also hides the progress bar. I want to keep the progress bar visible while still suppressing warnings, Do I need to modify the source code of the simulate_waveforms() to make this possible?

Below is some relevant code that I hope will help you know my issue.

simulate!(sim, refinement_limits = [0.3, 0.1, 0.05], 
max_tick_distance = 0.2u"mm",convergence_limit=1e-6,
max_distance_ratio = 4,verbose=true)

mc_events_clustered = @time SolidStateDetectors.cluster_detector_hits(mc_events, 0.1u"mm")
hits = ungroup_by_evtno(mc_events_clustered)
mc_events_per_det = group_by_evtno_and_detno(hits)
issorted(mc_events_per_det.detno)
mc_events_by_det = Table(consgroupedview(mc_events_per_det.detno, Tables.columns(mc_events_per_det)))
mc_events_det1 = Table(mc_events_by_det[1])

det_material = sim.detector.semiconductor.material

mc_events_fnoise = add_fano_noise(mc_events_det1, det_material.E_ionisation, det_material.f_fano)

filtered_events = mc_events_fnoise[findall(pts -> all(p -> CartesianPoint(T.(ustrip.(uconvert.(u"m", p)))) in sim.detector, pts), mc_events_fnoise.pos)];

MSGe_cartesian.zip

Thank you and looking forward your reply.

[fhagemann]

1. You get this error because, even though the point is in the detector, it is not in the semiconductor.
   In cylindrical coordinates, your point is at a radius of 0.014500043370356363, which is slightly outside of the 14.5mm radius that you gave the lower part of the semiconductor (not detector). It is in the Anode contact.
   You can avoid this by defining a bigger semiconductor that includes the Anode contact volume (so not as a union of two tubes, just a tube with a radius of 15.0mm and a height of 10mm).

 geometry:
     tube:
        r: 
          from: 0.0
          to: 15.0
        h: 10.0
        origin: [0,0,5]

2. Looks to me like the energy column of the Table is a Vector{Vector{...} and not a VectorOfVectors{...}.
   How do you read in your MC event information to the Julia session?
   If you delete the two strings, you don't save the output to a HDF5 file but just have it in the session.
   If that fits your purpose, you don't have to worry about this.

3. The charge drift itself does not yet support multithreading. What can slow the simulation down is if you calculate the waveforms for contacts that you are not interested in. If you set their weighting potentials to missing (e.g. sim.weighting_potentials[5] = missing if you do not care about the waveforms of the 5th contact), this might speed it up a bit.

4. Maybe instead of @suppress_err, there is another macro that gets rid of warnings, but keeps the progress bar?
   You could comment out the warning messages from the source code, but I think there should be a simpler way using Suppressor.jl

[zql2021]

1. I have defined a larger semiconductor that include the Anode contact volume, and it indeed worked. However, I am curious that why SolidStateDetectors allows the semiconductor and contact to overlap(or include)? I initially thought that the semiconductor only comprised the crystal (sensitive volume).

2. According the legend-julia-software-tutorial.jl, I read in my MC information by:

mc_events = open(read, G4output_filepath, Geant4CSVInput)
mc_events_clustered = @time SolidStateDetectors.cluster_detector_hits(mc_events, 0.1u"mm")
hits = ungroup_by_evtno(mc_events_clustered)
mc_events_per_det = group_by_evtno_and_detno(hits)
issorted(mc_events_per_det.detno)
mc_events_by_det = Table(consgroupedview(mc_events_per_det.detno, Tables.columns(mc_events_per_det)))
mc_events_det1 = Table(mc_events_by_det[1])
det_material = sim.detector.semiconductor.material
mc_events_fnoise = add_fano_noise(mc_events_det1, det_material.E_ionisation, det_material.f_fano)
filtered_events = mc_events_fnoise[findall(pts -> all(p -> CartesianPoint(T.(ustrip.(uconvert.(u"m", p)))) in sim.detector, pts), mc_events_fnoise.pos)];

And the energy column in filtered_events is indeed Vector{Vector{...}

I don't understand why :

import HDF5
using Suppressor
@suppress_err contact_charge_signals = simulate_waveforms(
        filtered_events[1:1000],
        sim,
        max_nsteps = 4000, 
        Δt = 1u"ns", 
        number_of_carriers = 100, # new
        diffusion = true, # new
        self_repulsion = true, # new
        verbose=true);

can worked normally, but adding output_dir and output_base_name will cause ArgumentError: Iterates of the argument to Flatten are not known to have constant length
I really need to save the waveform data in a file.

3.4 I will try the approach you mentioned.
Thank you for your help and your insights are invaluable to me.

[fhagemann]

1. Sometimes, it is hard to distinguish between a semiconductor and a contact. Even in your detector, it seems like the Anode contact is a Lithium-drifted contact that "extends" into the semiconductor.
   For the calculation of the electric field and the weighting potentials, it makes no difference if the semiconductor and the volumes overlap.
   This is because first, all grid points are marked as "semiconductor" if they fall into the semiconductor volume, and afterward the point types are updated to "fixed potential" if they fall within a contact.
   However, in the charge drift simulation, we only define the charge drift models for the semiconductor. Therefore, if a point falls outside the semiconductor, SolidStateDetectors.jl does not know how to drift the electrons and holes and throws an error.
   In the SSD simulations, I would always recommend having contact volumes with a non-zero thickness to overlap with the semiconductor.
2. simulate_waveforms has two methods that are written completely independently: one to keep the output in the Julia session here and the other one writing the results to a file here. The error for the one writing the results to HDF5 arises from this line: @info "-> $(length(flatview(mcevents.edep))) energy depositions to simulate."

flatviewis a function defined for VectorOfVectors{...} but not for Vector{Vector{...}} which causes the method to fail. This is especially unfortunate because it only fails to display an @info - not even because of the charge drift simulation.
   As a quick fix, you can change the line in your code that creates mc_events_det1 to this

   mc_events_det1 = let t = mc_events_by_det[1]
       Table((; [(cn, let c = t[cn]; c isa AbstractVector{<:AbstractVector} ? VectorOfVectors(c) : c end) for cn in fieldnames(t)]...))
   end

   such that the edep column becomes a VectorOfVectors, but we might change the source code to make this work in general..
   Thanks for pointing that out!

[zql2021]

1. Thank you for your detailed explanation，I have a deeper understanding of semiconductor and contact in the simulation.
   2.The code snippet you provided worked, now I successfully save the simulated waveforms in a file.
   Thank you very much,and best wishes!