CrystFEL issueshttps://gitlab.desy.de/thomas.white/crystfel/-/issues2023-05-10T10:37:23+02:00https://gitlab.desy.de/thomas.white/crystfel/-/issues/81Remove all simulation code2023-05-10T10:37:23+02:00Thomas WhiteRemove all simulation codeThis is a proposal to completely remove all the simulation code (`partial_sim` and `pattern_sim`) from CrystFEL. The rationale is as follows.
* The programs haven't had any new features added for over half a decade.
* In multiple rece...This is a proposal to completely remove all the simulation code (`partial_sim` and `pattern_sim`) from CrystFEL. The rationale is as follows.
* The programs haven't had any new features added for over half a decade.
* In multiple recent versions, it's been broken. This was the case in 0.10.0 and 0.10.1 for both programs (see c165bddef859284d31acae1f4a70d39d1f8e527e and https://gitlab.desy.de/thomas.white/crystfel/-/commit/5d267fe61871e88a5f69c8e5e5d4217fc70f58c6), and also for earlier versions.
* The programs are not well-used, so no-one seemed to notice the breakage until much later. It was fixed, only to break again later.
* There are better solutions for simulating patterns.
* The close integration with CrystFEL geometry files is no longer very meaningful. With early (2009-2012) versions, one could use the same geometry file for simulation and processing. Nowadays, many complicated file formats are possible. It's very difficult to write a file that will be directly readable: `image_hdf5_write` is horribly complicated, used *only* by pattern_sim, and [doesn't even work](https://gitlab.desy.de/thomas.white/crystfel/-/issues/6).
* The code in `pattern_sim` is completely separate and specific to that program. It even has its own reflection array structure. The program therefore doesn't benefit much from being part of the suite.
* The state of the art in data processing has reached a point where simulations aren't really needed any more. Even advanced algorithms nowadays have a good change of working on experimental data. Very clean experimental data is available where necessary.
* The build system stuff needed to support OpenCL has caused difficulty for people. This led to its being disabled by default (see 5c31874f20a37c2ac8d4c27d982c0783db990105).
* The `DataTemplate` paradigm (`filename + opaque DataTemplate ---> image + transparent detgeom`) doesn't work very well for simulation. The necessary routine (`image_create_for_simulation`) is often abused.
Someone wanting to simulate some patterns, these days, is honestly better off using a version of CrystFEL from about 2013. If simulation came back into demand, a new program, much simpler, could be written based on the old code, that writes only a specific file layout (suggestion: it could produce a geometry file as a side product).
Work in progress on branch [big-cleanup](https://gitlab.desy.de/thomas.white/crystfel/-/tree/big-cleanup). The immediate code reduction is nearly 6000 lines!CrystFEL 0.11.0Thomas WhiteThomas Whitehttps://gitlab.desy.de/thomas.white/crystfel/-/issues/78partial_sim is broken by DataTemplate2023-02-10T16:26:00+01:00Thomas Whitepartial_sim is broken by DataTemplatepartial_sim doesn't pay any attention to the command-line wavelength and bandwidth, and probably other things as well. I think it got broken in the update to DataTemplate.
pattern_sim is probably broken in the same way.
Since this woul...partial_sim doesn't pay any attention to the command-line wavelength and bandwidth, and probably other things as well. I think it got broken in the update to DataTemplate.
pattern_sim is probably broken in the same way.
Since this would have happened in 0.10.0 and no-one noticed until now, perhaps the best solution is just to remove all the simulation code.CrystFEL 0.10.2