Muscle X¶
What is Muscle X?
The Muscle X Application Suite is a collection of programs intended to assist with analyzing diffraction X-ray images.
Features
- Equator (eq)
- Quadrant Folding (qf)
- Projection Traces (pt)
- Scanning Diffraction (di)
- Image Merger (im)
- Diffraction Centroids (dc)
- DDF Processor (ddf)
Support
If you are having issues, please let us know. Submit feedback at https://github.com/biocatiit/musclex/issues.
How to Contribute
- Issue Tracker: https://github.com/biocatiit/musclex/issues
- Source Code: https://github.com/biocatiit/musclex
How to Cite
Click the following badge.
License
The project is licensed under the IIT License.
Copyright 1999 Illinois Institute of Technology
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ILLINOIS INSTITUTE OF TECHNOLOGY BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Except as contained in this notice, the name of Illinois Institute of Technology shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization from Illinois Institute of Technology.
Installation¶
We provide three ways to install Muscle X. See the following parts for details.
Instructions
Installation by Installer¶
For users who do not want to play with Python or Docker environment, we provide the pre-built installer for stand-alone MuscleX program.
Windows¶
Install through Windows Installer on Windows (64-bit).
Run¶
Find the shortcut of MuscleX on the Desktop or in the Start Menu, click it to run.
Some sample image data are installed to your Documents directory under folder MuscleX. Feel free to use them.
Mac OS X¶
Download the DMG file built for Mac OS X.
- Open the DMG file
- Drag the App Bundle MuscleX to the folder Applications
- Eject the image
You can always find the app in your Applications and click it to run.
python logo
Installation by Pip¶
In order to install the program, you need to install python 2.7 or 3.6.
Preparing¶
There are some modules need to be installed before installing Muscle X. They are Pip, OpenCV, PyQT4 or PyQT5, gfortran, PyFAI and Cython.
Linux¶
sudo apt-get update
sudo apt-get install python-pip libjpeg-dev libopencv-dev python-opencv python-qt4 cython pyfai gfortran
Mac OS¶
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install opencv
brew install cartr/qt4/pyqt
brew install gcc
sudo easy_install pip
sudo pip install pyfai
sudo pip install cython
Windows¶
For windows, we have built the Fortran source for you, so Fortran compiler is not required. But C compiler is still required to build C extensions during the installation process, and Microsoft Visual C++ (MSVC) is recommended.
Python 2.7 on Windows¶
The default MSVC chosen by Python 2.7 is MSVC 9.0. We recommend forcing the use of MSVC 14.0 instead. Follow the steps below.
- Uninstall Microsoft Visual C++ Compiler for Python 2.7 (MSVC 9.0) If Microsoft Visual C++ Compiler for Python 2.7 is installed, uninstall it (See Repair or remove programs in Windows 10).
- Install Microsoft Build Tools 2015 Update 3 (MSVC 14.0) The minimal customized installation of Microsoft Build Tools 2015 Update 3 is not enough. Make sure Windows SDK (8.1 or 10) is included.
- Temporarily set environment variable
VS90COMNTOOLSType the following command in your current cmd prompt:set VS90COMNTOOLS=%VS140COMNTOOLS%
Then install dependencies:
pip install numpy cython python-qt5 opencv-python pyfai
Python 3.6 on Windows¶
The default MSVC chosen by Python 3.6 is MSVC 14.0. The minimal customized installation of Microsoft Build Tools 2015 Update 3 is enough. Then install dependencies:
pip install cython numpy pyqt5 opencv-python pyfai
Note
Omit keyword sudo in the commands in following parts when working on Windows.
Running a program¶
Simply run
musclex [program shortcut]
For example, run this command to run Diffraction-Centroids
musclex dc
If you get ImportError: C extension: iNaT not built. If you want to import pandas from the source directory, you may need to run ‘python setup.py build_ext –inplace –force’ to build the C extensions first while you’re running the program, please re-install pandas by
pip install -U pandas
Updating¶
sudo pip install -U musclex
if you do not need to update other dependencies, you can run
sudo pip install -U --no-deps musclex
docker logo
Installation on Docker¶
In order to install the program, you need to install Docker first.
Installing Docker¶
To install Docker follow the docker website instructions and post-install instructions.
For Ubuntu:
sudo apt-get update
sudo apt-get install curl
sudo curl -sSL https://get.docker.com/ | sh
For Windows: you might need to do more steps as described here
Create docker group and add a user¶
You need to add a user to docker group in order to give docker run permission to the user
sudo groupadd docker
sudo usermod -aG docker [username] # replace [username] with your user name
Installing and updating Muscle X program suite¶
You need to have the muscle.sh script available in order to update, and run a Muscle X program on docker. You can download muscle.sh by
wget https://raw.githubusercontent.com/biocatiit/musclex/master/musclex.sh && chmod +x musclex.sh
Running a Muscle X program¶
Running the program by running musclex.sh following by a program shortcut
./musclex.sh [program shortcut]
For example, run this command to run Equator
./musclex.sh eq
If you are using ssh to log into a remote server to run musclex there use the following instead
./musclex.sh -s eq
To see previous versions
./musclex.sh -l
To run a specific version
./musclex.sh -v 1-14.11 eq
To update musclex
./musclex.sh -u
To see all options
./musclex.sh -h
Tests Overview¶
Testing a MuscleX Installation¶
Tests are provided to ensure results from an installation match expected results across different devices and dependency versions. Tests can be run from the command line or from the GUI.
GUI Testing¶
A testing module is provided in the launcher GUI. If no log is found, this procedure will run the first time the GUI is launched on a device. To use the GUI testing module,
- Launch the MuscleX GUI
- Click Run Tests in the bottom right corner
- Click Run Tests again at the top of the window to run all module tests
- If you’d only like to run the GPU tests, click run GPU Test and the test will determine if
pyopenclis installed properly and if MuscleX can access a GPU
Command Line testing¶
Open a terminal window and run musclex test to run all tests. Output from tests will be printed to the command line. Use musclex test_gpu to only run the GPU tests.
Testing Methodology¶
The testing suite in musclex_test/tests/ is used for verifying that MuscleX produces predictable results across version changes. Each module is tested independently using module_test.py. The module test runs in two modes - testrecord and testverify. In testrecord, module output is serialized to Pickle files for a particular set of input test data. In testverify, the same data is processed again and serialized to Pickle files, but is then compared to the files produced in testrecord mode. If any of these comparisons fail, it’s deemed a failing test and the location of the failure is returned to the user.
This approach provides a couple of important assurances. For developers, it ensures that no unexpected changes in module behavior come up that are caused by changes across versions. For installers, this verifies that an installation is behaving in the way developers expected. The testing framework also provides a tool for ensuring the reproducibility of results across devices.
testrecord¶
When a new version is pushed that causes changes in test output, a new set of test data should be recorded. To do this starting from the root of the repository:
- Change directory to the test directory.
cd musclex_test/tests/
- Run
module_test.pyintestrecordmode.
python module_test.py testrecord
Serialized Pickle files will be written to <modulename>/test_pickles. One file is written for each field in a module object’s info class variable.
-
An example of testrecord output.
testverify¶
Once output data has been recorded using testrecord, tests can be run using testverify. testverify mode processes the same input data as testrecord, and compares the output against the data in <module_name>/test_pickles.
To run the whole testing suite use the following command from the root of the repository:
python -m unittest discover -s musclex_test
A successful test should produce output similar to the following:
-
Testing Data¶
Test data used is found in /musclex_test/tests/test_images. Each time data in this directory is changed, testrecord should be rerun.
Given a module and corresponding settings, test Pickle files are written to folders in <module_name>/test_pickles_<settings_name>. In testverify mode, a directory <module_name>/tmp_verify_<settings_name> is utilized to store generated Pickle files that are used for comparison, but are deleted at the end of the test. A keeppickles argument is included in the module_test function and can be changed to True to keep the testverify Pickle objects. If a test fails, these temporary Pickle objects are kept to be used for comparison.
Test Suite Summary¶
The unittest suite is in musclex_test/tests/test_suite.py. It uses Python’s unit testing framework to run tests for each module under different configurations and provides a summary of the results. They can all be run using the instructions for running in testverify mode given above. A summary of each test that’s defined in the unittest suite is given below.
These default test settings were taken from examples in the tests folder in the root directory.
Equator Image Test - testEquatorImage¶
This test case runs a testverify pass for the following settings configurations.
settingsA = {
"left_sigmac" : 1.0,
"right_sigmac" : 1.0,
"orientation_model" : 0,
"nPeaks" : 2,
"model" : "Gaussian",
"isSkeletal" : True,
"mask_thres" : -1.0,
"90rotation" : False,
"blank_mask" : False
}
settingsB = {
"left_sigmac" : 1.0,
"right_sigmac" : 1.0,
"orientation_model" : 0,
"nPeaks" : 5,
"model" : "Voigt",
"isSkeletal" : True,
"mask_thres" : -1.0,
"90rotation" : False,
"blank_mask" : False
}
Data input: test_images
Pickles to compare to: eq/test_pickles_settingsA, eq/test_pickles_settingsB
To run only this test:
python musclex_test/tests/test_suite.py MuscleXTest.testEquatorImage
Quadrant Folder Test - testQuadrantFolder¶
The following settings are used.
settingsQF = {
'bgsub' : 'None',
'sigmoid' : 0.0,
'no_cache' : True,
'orientation_model' : 0
}
Data input: test_images
Pickles to compare to: qf/test_pickles_settingsQF
To run only this test:
python musclex_test/tests/test_suite.py MuscleXTest.testQuadrantFolder
Diffraction Centroids Test - testDiffractionCentroids¶
The following settings are used.
settingsDC = {
'orientation_model' : 0,
'90rotation' : False,
'no_cache' : True
}
Data input: test_images
Pickles to compare to: dc/test_pickles_settingsDC
To run only this test:
python musclex_test/tests/test_suite.py MuscleXTest.testDiffractionCentroids
Projection Traces Test - testProjectionTraces¶
The following settings are used.
settingsPT = {
'boxes' : {'box1' : ((200, 800),(500, 600))},
'bgsubs' : {'box1' : 0},
'types' : {'box1' : 'h'},
'peaks' : {'box1' : [100]},
'bgsub' : 'None',
'sigmoid' : 0.0,
'no_cache' : True,
'orientation_model' : 0
}
Data input: test_images
Pickles to compare to pt/test_pickles_settingsPT
To run only this test:
python musclex_test/tests/test_suite.py MuscleXTest.testProjectionTraces
Scanning Diffraction Test - testScanningDiffraction¶
Default settings are used (an empty settings dictionary is used to initialize processing).
Data input: test_images/di_test_data
Pickles to compare to di/test_pickles_settingsDI
To run only this test:
python musclex_test/tests/test_suite.py MuscleXTest.testScanningDiffraction
HDF Read Test - testHDFRead¶
This test checks to make sure h5py can read data from the HDF file generated
by Scanning Diffraction.
Data input: test_images/di_test_data/test.hdf
Pickle to compare to test_images/hdf_record/hdfdata_record.p
pyFAI Integration Test - testGPUIntegratePyFAI¶
Verifies that the pyFAI integration function works properly when using the csr_ocl integration method. Even if pyopencl is not installed, this function should work with or without a GPU.
GPU Device Test - testOpenCLDevice¶
Attempts an import of pyopencl and, if successful, lists the GPU devices available. This test passes if OpenCL and GPU acceleration is available and fails otherwise. Failing this test could indicate a problem with the pyopencl installation or imply that GPU acceleration is not available on your device.
Application Suite¶
Equator¶
Introduction¶
The purpose of the Equator program is to analyze the equatorial portion of muscle X-ray diffraction patterns.
The Equator program is designed to:
- Determine the inter filament lattice spacing, d10
- Fit Voigt or Gaussian model functions to the diffraction peaks in order to estimate their integrated intensities
- Determine I11/I10 intensity ratios
- Obtain estimates for σd and σs from the peak widths
screenshot
The program does this with as little user intervention as possible in order to improve reproducibility , reduce operator bias and increase efficiency. It can operate on a whole directory of images and produce results in hours instead of many weeks required for manual processing. Not all patterns are amenable to this approach, however. Any failed cases are flagged for manual processing, either within the Equator program or using other manual approaches. Typically the program succeeds with ~90% of patterns showing diffraction.
The Equatorial Diffraction Pattern from Striated Muscle¶
A good introduction to X-ray diffraction of vertebrate muscle can be found in Chapter 2 of Squire, JM, The Structural Basis of Muscular Contraction, Plenum 1981. A treatment of X-ray diffraction from insect flight muscle can be found in Irving (2006) X-ray Diffraction of Indirect Flight Muscle from Drosophila (in Nature’s Versatile Engine: Insect Flight Muscle Inside and Out, J Vigoreaux editor Landes Biosciences, Georgetown, TX).
Figure 1
The equatorial pattern arises from the projected density of mass in the A-band of the sarcomere. The thick filaments are packed into a hexagonal lattice with thin filaments interdigitated between them either in the trigonal positions (vertebrate muscle - Figure 1A) or half way between adjacent thin filaments (many insect flight muscles - Figure 1B). The density of the filaments projected onto a plane therefore represents a two dimensional crystal with the hexagonal lattice points occupied by thick filaments. One can draw imaginary planes through the crystallographic unit cell (see Figure 1A for vertebrate muscle) corresponding to various values of the so-called Miller indices h & k. In figure 1A are drawn the lattice planes corresponding to the h=1, k=0 (separated by d10) and h=1, k=1 separated by d11). These lattice planes give rise to the two strongest pairs of X-ray reflections in the X-ray diffraction pattern, the 1,0 and 1,1 reflections respectively with intensities I1,0 and I1,1. In insect muscle, the two strongest pairs of reflections are the 1,0 and 2,0 reflections because of its different lattice geometry (Figure 1B).
Figure 2
Figure 2 shows the geometry of a muscle diffraction experiment. The muscle sample is separated from the detector by a distance L. The spacing between thick filaments is ~ 40 nm, large compared to the wavelength of X-rays (~0.1nm). Bragg’s Law, n λ = 2dsinθ, describes the relationship between the lattice spacing d and the scattering angle 2θ. If d is large, θ will be small. L therefore is typically 2-3 m so that the distance S10, the distance from the center of the pattern to the 1,0 reflection, is of the order of a few mm. The scattering angle, 2θ, can be calculated from tan -1 (S10/L). At small angles however, tan2θ ≅ sin2θ ≅ 2θ. Substitution into Bragg’s law and solving for d10 gives a simplified expression: d10= nλL/S
It should be clear from this analysis that the lattice spacing is inversely proportional to S10. In skinned cardiac muscle the 1,1 and 1,0 equatorial reflections are often the only reflections visible. In skeletal muscle additional diffraction peaks past the 1,1 are frequently visible corresponding to other, higher order diffraction planes in the crystallographic unit cell denoted by larger values of h and k. In vertebrate muscle the first 5 such equatorial reflections are the 1,0; 1,1; 2,0; 2,1 and 3,0. In insect flight muscle, one can observe as many as 20 equatorial reflections. In addition, in skeletal muscle there is a peak located between the 1,0 and 1,1 reflections coming from the insertion of the thin filaments into the Z-band and is called the Z-band reflection. The position of this reflection from the center of the pattern is about 1.46 x S10 (Yu et al., 1977 J. Mol. Biol. 115:455-464). In general, the positions of the A-band reflections, excluding the Z-band reflection, obey a hexagonal lattice selection rule where S(h,k) = sqrt( h2+k2+h*k)*S10. Thus, d10 may be calculated from any equatorial reflection and applying the hexagonal selection rule. The d10 lattice spacing can be converted to inter-thick filament spacing by multiplying d10 by 2/√3).
The intensities of the 1,0 and 1,1 equatorial reflections may be determined from one-dimensional projections along the equator. When cross bridges bind to the thin filaments in vertebrate muscle there are radial and azimuthal movements of the crossbridges so there is a loss of mass on the 1,0 planes (containing only thick filaments) and a gain of mass on the 1,1 planes (Huxley, H. E. 1968. J. Mol. Biol. 37:507-520. and Haselgrove, J. C., and H. E. Huxley. 1973. J. Mol. Biol. 77:549-568). As a consequence, the intensities of the 1,0 reflections decreases and the 1,1 increases. I11/I10 intensity ratios, therefore, can be used to estimate shifts of mass from the region of the thick filament to region of the thin filament. Graded levels of isometric force
There is additional information in the equatorial patterns in the form of the widths of the diffraction peaks. It has been shown, Yu et al., 1985 Biophys J 47 :311-321, that the width of the Gaussian or Voigtian functions used to approximate the shape a given peak σh,k may be expressed (Yu et al., 1985 Biophys J 47 :311-321, Irving & Millman 1989) J Muscle Res Cell Motil. 10:385-94) as √(σc2+σd2Shk2+σs2Shk2), where Shk=√(h2+k2+hk)*S10 as above. σc is the known width of the X-ray beam, σd is related to amount of heterogeneity in inter-filament spacing among the myofibrils, and σs is related to the amount of paracrystalline (liquid-like) disorder of the myofilaments in the hexagonal lattice. Liquid-like disorder is sometimes called disorder of the second kind (Vainshtein, B. K. 1966. Diffraction of X-rays by Chain Molecules.Elsevier, Amsterdam.). σd can be expressed relative to Δd10/d10 as a measure of the width of the distribution of lattice spacings between the different myofibrils in the sample. σs can be expressed in terms of ΔX/d10 where ΔX is the standard deviation in the distribution of distances between nearest-neighbor unit cells in a given myofibril. σs increases substantially during contraction in skeletal muscle (Yu et al., 1985 op. cit.). Because of the dependency of the peak width on the square of the scattering angle, liquid like disorder rapidly causes the peaks to become indistinguishable from the background at higher scattering angles. The muscle lattice can also exhibit disorder of the first kind (Vainstein, 1966 op. cit.) which is the tendency of objects to vibrate isotropically around the lattice positions due to thermal energy resulting in a linear decrease in diffracted intensities with increasing scattering angle. This is often called “temperature factor type disorder” in crystallography. There is no direct way to estimate the degree of this kind of disorder without invoking a model structure.
How it works¶
When an image is selected, if there is an image in the same folder named “calibration.tif”, the calibration settings window will pop up (How to set calibration settings will be explained in the next section). If there’s no calibration image or it’s set already, the image will be processed automatically with the default parameters. However, if the image has ever been processed with the same version of the program, the cache will be downloaded, so the image won’t be processed again.
Image processing and fitting model¶
The program will process an image by going through multiple processes in the order
1. Find Center¶
4. Calculate Box Width¶
The image will be rotated by the rotation angle determined above, and the area inside R-min removed as in this image. The image will be cropped by using R-min x 1.5.
-
Then, the program will find the horizontal histogram from this image, calculate the background assuming a convex hull function to the histogram, and the integrated area will be selected as between the start and end point of the histogram. From the image above, the horizontal histogram was
-
5. Get Intensity Histogram¶
When integrated area is calculated, the program will produce a histogram from the rotated image inside the integrated area. (If the blank image and mask is set, the original image will be subtracted by the blank image before rotation)
-
6. Apply Convex hull to intensity histogram¶
The original histogram will be split into left and right sides. Then, the convex hull to each half pattern will be calculated in order to remove the background by using R-min as a starting point.
-
7. Find Diffraction Peaks¶
The program will find peaks from left and right histograms which have had the convex hull background subtracted. This process will find all locations of the peaks. If the image is noisy, it is possible that the program will find too many peaks. ( In the image below, the program found only 2 peaks because the image was not noisy, so it worked very well )
-
8. Managing Diffraction Peaks¶
This process prepares and corrects the background-subtracted, intensity histogram before fitting a model. It is possible that the program found peaks from noise, misplaced peaks, or both false positive peaks and false negative peaks. Therefore, these peaks need to be re-positioned in case this process fails. The algorithm will try to find first symmetric peaks, and find S10 which is the distance from center to the first peak on the equator. After that, all peak locations will be calculated by using S10 and theta(h,k). The number of peaks on each side to be fit needs to be specified by the user (default and minimum is 2)
9. Fit Model¶
The program will fit the model to the histogram by using the specified model (currently Gaussian and Voigtian models are supported), initial S10,and the area of the reflection peaks. Finally, we will obtain the new parameters we want from the fitting results. This will include the area of each peak, S10, sigma D, sigma S, gamma, and I11/I10. However, if you see only 4 reflection from the pattern (Only I11 and I10 on each side), it will be overdetermined if you use Voigt as a fitting model. To solve this problem, sigma S or gamma should be fixed.
If there are some parameters that need to be configured manually to obtain good fits, the program will run these processes again, but it will not start from the beginning. Instead, it will start from the process after the manual one. For example, if the Box Width is set manually, the program will run processes from Get Intensity Histogram to Fit Model because the center, rotation angle and r-min do not need to be recalculated.
How to use¶
Once the program run, you will see an input file dialog, so you can select the file you want to process. If you want to process multiple files in a folder, please select a file in that folder, and click “Process Current Folder”.
When the processing window opened, there are 3 tabs on the top,
Image¶
In this tab, you will see the selected image, Display Options section, and Image Processing section, Reject check box, next button previous button, and Process This Folder button.
imagetab
Display Options¶
In Display Options section, you are able to select what you want to see in the image. All check boxes are all about displaying information in the image. In image above,
- Center : blue dot
- Integrated Area (Box Width) : green lines
- R-min : red dotted circle
- Histogram : white plot as original histogram and red line as fitting model
- Peaks : yellow lines
display_options
You can also set min and max intensity for displayed image in this section. To zoom the image, you can click “Zoom In” button and select the zoom in area in the image, or using mouse wheeling in the image directly. This options in this section will not affect the image processing.
Image Processing¶
In Image Processing section, you will see multiple buttons allow you to calibrate the image and set some properties manually.
After “Calibration Settings” is pressed, there’s a window popped up. This window will allow you to select the calibration image by clicking on Browse or setting parameters manually. See Calibration Settings for more details
Before setting manual rotation and center, it’s better to zoom the image to the area of the diffraction because it will be easier to set these parameters correctly. To set the rotation and center, you need to click 2 positions of the image. The first one will be a reflection peak on one side of the equator, and the second one will be the corresponding (opposite) reflection peak on the other side of the equator. To cancel, press ESC.
-
This assumes that the center of diffraction is correct. After the button is clicked, the program will allow users to select an angle by moving a line. Clicking on image when the line is on the equator of the diffraction will set manual rotation angle. To cancel, press ESC. (Make sure that “Fixed Angle” is not checked)
-
After the button clicked, you will see the red circle when you move the cursor around. To set manual R-min, please click on image when the circle size is the size of R-min you want. To cancel, press ESC.
-
To set the integrated area, you need to click 2 positions of the image. The first one will be the start line, and the second one will be the end line. To cancel, press ESC.
-
This option is available on version 1.6 or upper. Click Here to see more details
This feature will allow users to fixed rotation angle for every image which has not been processed. After the checkbox is checked, the manual rotation angle which is set by buttons will be ignored.
This feature works in the same way as fixed angle, but the fixed value will be start and end position (pixel) of the box
Fitting¶
In this tab, you will see the graph produced from the intensity histogram in the integrated area, and fitting information.
-
General Settings¶
In the section, you are able set necessary parameters for fitting process including skeletal muscle checkbox, number of peaks on each side, and fitting model. The model functions for the peaks currently supported are Voigtian and Gaussian. If the Number of Peaks selected are more than the number of peaks, the program still tries to fit the model with the selected Number of Peaks.
-
In cases where the program misplaces the peak locations, you can do peak selection manually by clicking on “Start Manual Peak Selection”. After the button clicked, you can select the how many peak locations you want by clicking on the graph. However, it’s sufficient to just select S10 of left and right side. Click “Done” when you are done. To cancel, press ESC.
-
Display Options¶
In this section, you will be able to select what you want to see in the graph. All check boxes are all about displaying information in the graph. In the image above,
- Original Histogram : black line
- After Convex hull : green line
- Peaks : red lines
- Fitting Graph (Best fit function graph) : blue line
- Z line : yellow line
- Center X : magenta line
-
You can also zoom-in to see more detail by the same mechanism as in Image tab. All options in this section will not affect the fitting results.
Settings¶
In this section, once a setting is changed, the fit will be recalculated. You can manually configure and lock all parameters of fitting model including Sigma C, Sigma S, Sigma D, gamma and skeletal parameters. These parameters are independent on each side, so you have to choose the side by choosing left or right tab before setting these parameters. Press “Re-fitting” when you want to re-fitting will the new settings.
-
Diffraction data taken on integrating detectors such as CCD detectors will have read noise. This can result in a constant offset to the background subtracted diffraction peaks from zero. This can lead to systematic error(“BAckgeos in your peak measurement. In this case it is possible to add a user selected constant offset (“Background K”) that is added to the fitting function prior to fitting the data. In practice this number is selected as the number that provides the best “eyeball” fit to the residual background. Since this residual background often has some residual structure that is hard to model, the constant is not adjusted by the fitting proces.s
Results¶
Important fitting results are shown in this tab. If the calibration parameters are set, the program will also show d10.
-
Blank Image and Mask¶
There 2 settings you can set here, blank image and mask
Blank Image¶
Once the window launches, you will see “Select Blank Image(s)” at the bottom. Then, you can select a blank image or multiple images. The program will average them all if you select multiple images. If the image is from Pilatus detector. The program will initial mask threshold, and paint the mask as orange color. You can change it in order to make correct mask.
- 
-
Additional Mask¶
The mask threshold can create mask from the threshold value. If you have additional area which you want to ignore, you can click “Draw Additional Mask”. This button will available only when the blank image is selected. After the button is clicked, there will be a new window pops up. This dialog is a toolkit from PyMca
-
Before drawing additional mask, you can zoom in by disable the arrow by just pressing it
-
You can also change color type or intensity range by pressing the palette icon, and flip the image vertically by pressing Mona Lisa icon. To draw the mask, there’re several options.
-
- Drawing by rectangle selection
- - Drawing by brush
- - Drawing by polygon selection
-
To clear the mask, press 
-
To erase some mask, press 
- and erase them from the image
Save¶
-
After every thing is set, and OK is pressed, the blank image and mask will be saved to settings folder which is created under original selected image directory (not the blank image). When this settings is launched again, these images will be downloaded.
Summary¶
Once the image is processed, the results of the analysis will be saved to a file called “summary.csv”. This summary file will be in the “eq_results” folder that is created as a sub-directory of the folder containing the selected file. Therefore, one folder will have one summary file.
-
Quadrant Folding¶
Introduction¶
The equator and the meridian of a fiber diffraction pattern divides a pattern into four quadrants. Because of Friedel’s Law each of the 4 quadrants will contain the same information. If you add the four quadrants together you effectively have 4 times the signal improving the signal to noise by factor of 2 (square root of 4). You can regenerate a full diffraction pattern by simply rotating the summed quadrant.
It usually better to analyze this “quadrant folded” image than the original image because of the improved signal to noise will make the data easier to fit as well as allow better estimates of the backgrounds. Quadrant Folding is a program for generating such a quadrant-folded image. Other advantages of quadrant folding is that it can compensate for detector imperfections by substituting data from the unaffected quadrants for the affected area. This is particularly useful for data collected using Pilatus detectors which have substantial gaps between the detector elements.
The program also can estimate and remove the diffuse background scattering from an image using a number of different algorithms, alone or in combination. It includes all the background subtraction routines from the CCP13 suite (https://github.com/scattering-central/CCP13) plus one based on a two dimensional convex hull and another on white top hat filtering. It is possible to merge different background estimates to apply one algorithm at low scattering angles and a different one at high scattering angles.
Once set up, the program can process an entire directory of images without user intervention
-
More Details¶
How it works¶
When an image is selected, the program will immediately process the image automatically. The resulting image will be saved as [image_name].result.tif in the qf_results folder that is created as a subdirectory of the folder containing the selected file. The program will process an image by going through a series of processes in the order described below
Processes¶
1. Find Center¶
3. Calculate Average Quadrant¶
To find the average fold, the image will be split into 4 quadrants, and each quadrant is flipped to be in the same orientation as below.
-
After all quadrants are flipped, a new image will be generated by averaging each pixel from all quadrants. The mask threshold will be used to determine if that pixel will be included in averaging process. In this case, the mask threshold is 0. That means if the pixel intensity is less than 0, the program will ignore that pixel. Finally, the result image will look like the image on the right. The full 2D pattern can then be regenerated by rotations of this image.
-
4. Calculate R-min and R-max¶
For the background subtraction algorithms to work, it is necessary to define a minimum radius R-min and a maximum radius R-max to define the region that is assumed to contain real data. To find R-max, the radial histogram will first be re-computed from the average folded image. R-max (blue line) is initially chosen to be simply 60% of the diagonal of the image. These two values are intended as just approximate starting points, that users can change manually.
-
5. Apply Background Subtraction¶
Fiber diffraction patterns from muscle have a substantial diffuse background arising from non-crystalline parts of the muscle that do not diffract coherently including membranes and mitochondria as well as diffuse scattering from the myosin heads. Intact cardiac muscle, in particular has a very strong background. This background pattern has a complex structure making it hard to model and effectively remove. Various algorithms exist that attempt to remove the background. No one technique so far can effectively remove background in all parts of the image without over subtraction in some parts of the pattern but some techniques work better than others depending on the muscle and specific preparation. Quadrant folding implements several different algorithms that can be used individually or in combination that can be used to background subtracted images for display or subsequent analysis.
Generally speaking, the “White Top-Hat” algorithm works well at large radii from the center and some other algorithm will work better than others, depending on the type of muscle generating the X-ray pattern, at low radii. Two images will be produced from this process. The first image will be created using one of several background subtraction approaches, and the second image will be created using the White-top-hat approach. These 2 images will be merged as described below. At present, there are 4 options for background subtraction in the first (low radius) image, Circularly symmetric, 2D Convex hull, and White-top-hat.
-
The necessary parameters for this scheme for background subtraction are the pixel range in percentages, the radius bin size in pixels, smooth parameter for spline. In this method, the program will
- Go to each radius bin, sort all pixel in the bin, and calculate background value by averaging intensity of pixels in specified pixel range
- After the a background value is determined for each radius bin, these values are combined by fitting a spline curve with specified smooth and tension parameters
- A 2D background is generated by spreading the spline curve through 360 degrees. This 2D image can then be subtracted from the original image to generate a background-subtracted image
This method is from CCP13 FibreFix and their fortran codes are available on GitHub
-
The necessary parameters for this kind of background subtraction are again R-max and R-min. Initial values for these parameters are provided by the program. Users can change these parameters if necessary. The convex hull identifies the closest surface that contains all the points along the smoothly decaying background. A 1D convex hull can be generated using the “shrink wrap” algorithm that follows the surface generating a series of of lowest points along the surface. The 2D convex hull surface is created by generating a 1D convex hull background subtraction to the radially integrated intensity over a 0.5 angle bin for each bin in the 90 degree summed quadrant using R-min and R-max as start and end points for calculating the 1D convex hull. Prior to calculating the convex hull, this intensity trace is smoothed to reduce the effect of noise. To create a background image, the pchip algorithm is used to interpolate between points in each 1D convex hulls. The pchip algorithm will be used again to interpolate a surface joining the points on the 1D convex hulls to create a 3D background image. Then, the program will subtract the background image from the original image.
-
-
The necessary parameters for this kind of background subtraction are R-max, R-max, window size, window separation, pixel range, smooth and tension factors. The roving window background subtraction method of Paul Langan estimates the background by moving a window (of size input by the user) across the collected data. The pixel values within this window are sorted and those in the user-selected range are taken as background (except pixel values lying outside the pattern extents or specified by the user as values to discard). The average pixel value within this range is then assigned to be the estimated background at the centre of the window. Fiinally, a smoothing spline under tension is fitted to fill in the gaps between window centres. (From FibreFix Document)
-
The white top-hat transform is an operation that extracts small elements and details from given images. The white top-hat transform of f is given by Tw(f)= f – f°b where ° denotes the morphological opening operation by a structural element b which we will call a kernel. This will retain features that are “smaller” than the structuring element (i.e., places where the structuring element does not fit in), and are brighter than their surroundings. So it is somewhat similar to the “rolling ball” background subtraction algorithm in ImageJ but more general. The only parameter that needs to be specified for this kind of background subtraction is the size of the applied kernel. The kernel will be created by using the disk function from scikit-image, and applied to original image by using the white_tophat function.
-
The necessary parameters are R-min, R-max, number of cycles, and filter type. This method of background subtraction is based on that of M.I. Ivanova and L. Makowski (Acta Cryst. (1998) A54, 626-631). For more information please see CCP13 FibreFix
6. Merge Images¶
If two background images are generated, one at high radius from the center, one at low, they are merged together by defining a “merge gradient”. The merge radius (R-max) is the position at which the two background images are merged together while the “merge gradient” is a smoothness parameter to make the transition between the two images smooth.
-
Merge gradient is a parameter k for [[/images/QF/gradient.png]] while r is merge radius. Therefore, if the merge border will be smoother when k < 1. If k > 1, the merge border will be more pronounced.
7. Generate Result Image¶
To generate a result image, the program will just copy the background subtracted average folded image and flip them to correct their orientation for all 4 quadrants, and then combine them together.
-
How to use¶
After the program is opened, users will be able to select a file or a folder to process by pressing the button or select an option on the menu bar.
-
When an image is selected, the program will automatically process the image if it has not been processed before. There are 2 tabs in the program, Original image, and Results.
Original Image¶
In this tab, the original image will be shown along with various options displayed on the right, Display Options, Image Processing, and Fix Center.
Display Options¶
-
All options in Display Options will not affect any processing. These options allow users to see more detail in the image by setting minimal intensity, maximum intensity, and zooming. You can also choose whether or not to see the meridional and equatorial axes. To zoom in, the user needs to simply press the Zoom in button, and select the zoom region by drawing a rectangle as shown below.
-
Image Processing¶
-
Before setting manual rotation and center, it’s better to zoom the image to the area of the diffraction because it will be easier to set these parameters correctly. To set the rotation and center, you need to click 2 positions of the image. The first one will be a reflection peak on one side of the equator, and the second one will be the corresponding (opposite) reflection peak on the other side of the equator. To cancel, press ESC.
-
This assumes that the center of diffraction is correct. After the button is clicked, the program will allow users to select an angle by moving a line. Clicking on image when the line is on the equator of the diffraction will set manual rotation angle. To cancel, press ESC.
-
The Mask Threshold is used for excluding certain pixel values when calculating the folded image. The program will ignore pixels with intensity below mask threshold. This can be used to remove the dark gaps in images resulting from the gaps between detector elements in Pilatus detectors, as well as other detector abnormalities.
To fix the center position to a user supplied value, you can check Fix Center check box, specify the coordinates of the beam center (before rotation). The image will be reprocessed when x, or y is changed. This will affect the next image if it’s still checked.
Results¶
In this tab, the resulting image will be displayed along with options on the right. Display Options will provide the same options as in Original Image tab. Users can specify min and max display intensity, and zooming.
-
For the background subtraction section, by default, the program will not apply any background subtraction. To apply background subtraction, you can choose a method by method drop-down list. There are currently 6 options, Circularly Symmetric, 2D Convex hull, Roving Window, White-top-hat, Smoothed-Gaussian, and Smoothed-Boxcar
For each method, users can select R-min and R-max manually by pressing the button and select R-min and R-max in the image as shown below
-
Circularly Symmetric¶
-
2D convex hull¶
-
Roving Window¶
-
White top hat¶
-
Smoothed Gaussian¶
-
Smoothed Boxcar¶
-
You can change the parameters and click Apply to make the program re-process the background subtraction
Merging¶
For the merging settings, they will be under the white line in the Background Subtraction section. There is the ability to set also another top hat parameter for regions of the image outside the R-max so the top hat parameter can be different inside and outside the merge radius if desired.
To set the merge gradient, simply change the value in the Merge Gradient spinbox
Projection Traces¶
Introduction¶
Even in globally background subtracted images it is usually necessary to reduce 2d diffraction patterns to 1d projections in order to extract accurate position and intensity information from diffraction peaks. While it is possible to do these projections in FIT2D, there is a steep learning curve and its scripting capabilities while powerful, are not easy to use.
Projection traces was conceived originally as a program to extract the integrated intensity along a layer line in order to identify positions of intensity maxima along with their integrated intensities. It also allows measurement of the radial width of meridional reflections. These processes are complicated by the presence of residual background after global 2D background subtraction. The program is now also capable of doing intensity projections along and parallel to the meridian and saving the results as 1D intensity traces to ASCII files for analysis with other programs.
An important capability is to process whole directories of images without user intervention once the program is set up for the desired analysis.
If using a version prior to MuscleX 1.14.11, it is important that these images have been pre-processed using the Quadrant Folding program to ensure that the center of the diffraction pattern coincides with the center of the image (image width/2 and image height/2) with the meridian of the pattern parallel to the long axis of the image
This program is available on MuscleX version 1.3.0 or later.
-
More Details¶
How it works¶
When an image is selected, if the image has ever been processed with the same version of the program, the cache will be downloaded, so the image won’t be processed again. Users need to specify boxes for the desired integration area and initial peak positions to trigger image processing. If the boxes and peaks are specified for one image in a folder, all other images in the same folder will be processed using these boxes and peak positions.
Image processing¶
The program will process an image by going through a series of steps in the following order. (This assumes that the boxes and peaks have been specified already)
1. Obtain the 1-D Projection from a box¶
In this process users just sum all pixel intensities vertically or horizontally depends on the shape of the box
-
2. Fit Model¶
In this example, the intensity is integrated along a layer line and projected onto a line perpendicular to the meridian. The sharp peak in the center is a meridional reflection. To fit the model, you need to specify the approximate peak locations. This model will try to fit a Gaussian model to every peak and other 3 Gaussians at the center. These 3 Gaussians at the center are overall background, meridian background, and meridian peak. Therefore, the results will be as in the intensity trace shown below. The overall background is the blue area, the background under the meridian is the yellow area and the meridional peak is the red area. The green line is the resulting fit summing all peak Gaussians while the red lines are the positions of the layer line peak centers after fitting.
-
3. Get Background Subtracted Projection¶
We try to remove all background and center peak, so in this process, the program will remove them by using 3 Gaussians from previous step.
Basically, the program will just subtract the 3 center Gaussians (colored) from the original projection (black line).
-
Then, the result is
-
4. Calculate all peak information¶
The main objective of this process is to find centroids of all peaks. In order to calculate peaks’ centroids, the program needs to …
- Calculate baseline of each peak. By default, the baseline is the half-height of the peak, but you can change it manually later.
- Find the left and right intersection locations around the peak (black) and its baseline (yellow) as below
- - Calculate the centroid by calculate the dot product of x (distance from center) and y (projected intensity) in range of the left and right intersections. Then, divide the result by the sum of y (projected intensity) along the range.
How to use¶
Once the program is run, you will see the steps 1 to 4 that you have to do on the left menu bar.
-
First, select an image to process. Second, configure calibration settings (optional). Third, add how many boxes you want. Finally, select peak locations in each box. For each folder, you just need to do it once on the first image. When another image in the same folder is processed, all these settings will be used.
In this page, you will know about …
Adding Boxes¶
There are three options for box selection: axis aligned, oriented, and center oriented boxes.
To add a box, you have to click on one of the box selection buttons, and draw the box on the image. Axis aligned boxes are drawn by selecting a corner and dragging to from a box. The axis of projection can be selected as either horizontal or vertical. Oriented boxes are drawn by select a box center, then the length of the axis of projection, and finally the width of the box. Center oriented boxes are equivalent to oriented boxes, but the pivot is fixed at the center of the image (either the diffraction center, image center (if quadrant folded), or a manually chosen center).
-
After a box is drawn, a box detail dialog will pop up. You need to specify box’s name and background subtraction method along with the axis of projection for axis aligned boxes. This background subtraction will apply to the projection after peaks are selected. The axis of projection is assumed to be along the length of the box.
-
Once a box is added, there’s a new tab created. You can add how many boxes you need by repeatedly adding a box. The box name will be written on the image and its information will be displayed in its tab.
-
Select Approximate peak locations¶
To select the approximate peak locations, you can just click on the button, select them in the box and click “Done”. When you select a peak, the program will automatically select the corresponding peak on the opposite site.
-
If there are multiple boxes, it’s better to select the peak locations in the box tab. To select peaks in the box tab, you can go to that box tab and select peaks by pressing “Select Peaks” button, then select them on the 1-D projection on the left and press “Done”.
-
When peaks in a box is selected, the program will process image by these steps to get all results
Select Convex Hull Range¶
If you select Convex Hull as background subtraction method for a box, and peaks are selected. The program will automatically select start and end points for Convex Hull. If you want to change this range, you can click “Set Manual Convex Hull Range” and select start and end points on the plot
-
Remove A Box¶
To remove a box, you can just close its tab
Display options¶
In the image tab, there are display options shown on the right. These options will not affect any processing. You can check “Boxes”, “Center” or “Peaks” to be displayed on the image. You can zoom-in by pressing “Zoom in” and select the zoom in area on the image by drawing a rectangle. (You can zoom-in or zoom-out by mouse wheeling too). Also, you can select min/max intensity to see the image clearly.
As of version 1.14.11, you can specify a rotation angle and center manually. This is done by specifying two reflection peaks in the diffraction. The image will be rotated according to the new angle.
The Quadrant Folded? option determines whether the center is assumed to be the half width and height of the image if it’s been quadrant folded. If this box is unchecked, the center will be determined automatically. All peaks will be reflected over this center in any drawn boxes. Use the Center checkbox to see which center is used by Projection Traces.
-
In each box tab, you will see multiple check boxes in the section. You can check or uncheck whether you want to see it in the plot
-
Other Options¶
There’re several options on the bottom left in image tab.
-
- Export All 1-D Projections
If this checkbox is checked, and boxes are added, the program will save the original 1-D projection to a text file in 1d_projections folder under pt_results which is created under the image directory. If peaks are also specified, background subtracted projection will be saved in the same folder too. - Process Current Folder
This will process the whole images in current directory with current settings (boxes and peaks) - Previous and Next Buttons
This will make the program go to process the next or previous image with current settings
Scanning Diffraction¶
Introduction¶
Scanning diffraction imaging experiments attempt to determine the distribution of diffracting materials such as collagen, myelin, and amyloid structures as a function of position in a sample that is raster scanned in a microbeam. The diffracting materials at a given position in the sample will have varying degrees of orientation so that meridional reflections will be spread out into arcs whose angular width depends on the degree of orientation. In the isotropic case, these reflections would form complete rings. The information that is desired to extract from these images is 1) the total amount of diffraction material which will be proportional to the integrated intensity 2) chemical identity of the diffracting material which can be determined from its d-spacing 3) orientation of the long axis of the diffracting material (which direction the molecules are “pointing”) and 4) degree of disorientation around this action, determined by the angular width of the meridional arcs. A scanning diffraction imaging experiment may produce many thousands of images so there is a need to process these as automatically as possible with little or no human intervention. The Circular Projection program has been written in order to automatically detect individual diffraction rings and determine the structural parameters for this ring.
This information includes …
- The distance from center to the ring and the calculated d-spacing
- The standard deviation of the ring intensity distribution in the radial direction
- The integrated area of the ring
- The orientation angle of the ring
- The standard deviation of the intensity in the azimuthal direction i.e. the standard deviation of the orientation angle
-
-
More Details¶
How it works¶
There are 2 modes for Scanning Diffraction
Individual Image Mode¶
When an image is selected, if there is an image in the same folder named “calibration.tif”, the calibration settings the calibration settings window will pop up. If there’s no calibration image or it’s set already, the image will be processed automatically. However, if the image has ever been processed with the same version of the program, the cache will be downloaded, so the image won’t be processed again. Using a calibration image or inputting the calibration information directly allows conversion of distance in pixels to d-spacings in nm. If calibration information is not provided, distances to diffraction rings are left in numbers of pixels.
To process an image, the program will go through multiple processes in the order:
1. Find Center¶
2. Obtain 2D integrated intensities and 1D Radial Integration¶
In this process, the program will obtain 2D and 1D radial integrated intensities of the entire diffraction image using the previously determined center. The program will use functions from the pyFAI library to produce the integrated intensity using FPGA hardware acceleration if available. The 2D integrated intensity will be used for Finding Rings by Log Central Differences Method and 1D integrated intensity (after Convex Hull background subtraction is applied) will be used for Finding Rings by Partial Integration Method and Fitting Model.
For example, if the image is
-
The 2D radial integrated intensity map is ..
-
and the 1D radial integrated map is ..
-
3. Find Rings by Partial Integration Method¶
In order to find the ring locations, 2 methods are used, Partial Integration, and Central Difference. In this process, pyFAI will be used to find the 1D integrated intensity a pie-shaped region around the center. This angular range of this pie-shaped region can be specified by user with a default of 90 degrees. Here are shown multiple 1D integrated intensity traces, one from each range, when the angle size is 90.
-
For each range, the program will try to find peak locations in the integrated intensity trace. The program will consider a peak as representing a ring when a peak appears at least one quarter of the number of angular ranges. For example, if there are 8 ranges and a peak appears in 2 ranges, this peak is considered a ring. However, if a peak does not appear in many ranges, but its area is large, it’s may also be considered as a ring.
4. Find Rings by Log Central Differences Method¶
This can be useful at locating weak peaks in the presence of strong ones in a pattern. In this process, the program will try to find rings by finding “runs”. First, 2D integration produces a 2D plot of pixel intensity as a function of angle and radius. Two additional images are generated by shifting the radius axis first left and then right by q0 pixels and te three images summed. The result is shown on a logarithmic scale After that, the program will go to each column and find “run:s. Runs are defined as columns of pixels containing values higher than the median in the image, and those pixels have similar value. Then, the rings will be considered from grouping runs. For example,
- 2D integration
- - After applying log central differences
- - Finding runs (red lines)
- - Grouping runs to rings (white lines)
-
5. Merge Rings¶
After the ring locations are calculated by 2 methods above, those rings will be merged in this process. Rings with similar distances will be averaged. For example, if the first method found rings at 50, 80, 118 pixels and the second method found rings at 82, 120, 180 pixels. The final rings will be at 50, 81, 119, 180 pixels.
6. Fit Gaussian Models to the 1D Radial Integrated intensity¶
In this process, multiple Gaussians will be fit to the 1D radially integrated intensity trace by using the ring locations as the initial centers for the Gaussians.
-
After fitting, we will get ..
- Distance from center to the ring
- Standard deviation of the ring distribution in radial direction
- Integrated area of the ring
7. Finding Other Properties of Each Ring¶
In this process, the program will go to each ring and obtain the angular projection of the ring. In the example above, these are 2 angular projections where the x-axis is the angle in radians and the y-axis is integrated intensity.
-
Then, 2 Gaussians will be fit to the projection to find the angle, and the angular range. The distance between the 2 Gaussian centers is assumed to be Pi radians.
-
Finally, we will get …
- Orientation angle of the ring
- Standard deviation of the orientation angle (Angle sigma)
- Integrated area of the ring
After everything is processed, results will be written to summary.csv and a different rings.csv file. These files will be located in the folder cp results which is created under the folder containing the images. Currently, the file summary.csv contains the name of the image, the total diffracted intensity, and number of rings detected. In the rings.csv file, all information for all rings will be provided along with their fitting error. Note that there is no attempt made to index families of rings. Each ring is associated with its own unique d-spacing. The assumption is that the results data files will be processed by user supplied code to index the patterns.
Folder Mode¶
When a folder is selected, the program will get all image names in the folder, and check if all images have been processed by checking the summary.csv and rings.csv files. If there are some images which have not been processed, the program will process all those images using the individual mode first, and then generate maps from summary.csv and rings.csv.
In this mode, you will see 4 maps, Total Intensity Maps, Angular Range Maps, Orientation and Intensity Vector Field, and Elliptical Representation. Each pixel in the maps represents an individual diffraction image from the original image folder. To generate the maps, the program can use the hdf data file generated by the BioCAT scanning diffraction imaging script if it was used to produce the image files. If there is no hdf file, one can be generated by providing start point in the X-direction and the X-pixel size and the start point in the Y direction and the Y-pixel size.
1. Total Intensity Maps¶
Total intensity from summary.csv
-
2. Angular Range Maps¶
Value of Standard deviation of the orientation angle (Angle sigma) of the best ring from rings.csv
-
3. Orientation and Intensity Vector¶
Angle of the best ring from rings.csv as vector direction, and total intensity from summary.csv as color
-
4. Elliptical Representation¶
Angle of the best ring from rings.csv as ellipse orientation, total intensity from summary.csv as color, and angle sigma as ellipse size
-
These are the ring parameters which provides the lowest fitting error. The fitting error and angle sigma must be less than 1 in order for isotropic rings to be omitted from the angular distribution maps.
How to use¶
Once the program is run, you will see these 2 buttons for Individual Image Mode and Folder Mode/Batch Mode
-
Individual Image Mode¶
After an image is selected, the program will process it, and display all information in 4 tabs, Image, Method 1, Method 2, and Results
Image¶
-
There are several options on the right bar. In Settings section, you can do Calibration Settings and ring selections. Several methods of finding orientation based on GMM or Herman Orientation Factor (HoF) are provided. Region of Interest (ROI) is shown as a semitransparent red ring. It is used as the integration area for HoF method and only rings in it are considered when performing GMM fitting.
To do the ring selection, you have to click the button, pick rings on the image, and click Done. Setting ROI, R-min and R-max can be done in the same way.
-
Another section is Display Options. You can set max intensity and min intensity to see the rings clearly. Also, there are some check boxes that allow you to see ring locations and their properties.
Method 1¶
This method is the first method of finding rings process. Click Here to see how it works
-
In this tab, you will see the image on the left along with its partial radial integrations, and its 1D integration on the right.
The partial radial integrations are displayed in the graph on the top. This represents 1D radial integration for specific angle range. The title of the graph will tell you what the current range is and what peaks are found. In the image above, the current range is 0-90 degrees. The image on the left will cover this range with red color. To see other ranges, you can click left or right arrow. You can also change angle size by changing Range Angle value. This requires program to re-calculate everything.
-
The 1D radial integration for all 360 degrees will be display at the bottom with final peaks as its title
Method 2¶
This method is the second method of finding rings process. Click Here to see how it works
-
There is a drop down list on the top that you can select what to see. There are original 2D integration, Central Differences, and log of central differences. You can also see runs and rings by just checking the box. Runs will be displayed in red and rings will be displayed in white.
Results¶
In this tab, you will see all fitting results on both radial integration, and angle projection.
-
You can select fitting graph that you want to see by selecting the drop down list on the top. You can select peaks manually by pressing “Select Peaks Manually”, select peaks in the graph, and click “Done”.
-
For fitting results, there are 2 text boxes at the bottom of this tab.
The box on the left will display total intensity and fitting results of peaks in 1D radial integration. For each peak, you will see …
- Center of the peak : Distance from the ring to center of the diffraction.
- Alpha : Area underneath the peak.
- Sigma d : Standard deviation of the ring distribution in radial direction.
-
The box on the right will display fitting results of ring orientation in angular projection. For each ring, you will see …
- Angle : Orientation angle of the ring
- Range : Angle +- Sigma
- Sigma : Standard deviation of the orientation angle
- Intensity : Area underneath the fitting gaussian
- Fitting Error : fitting error for orientation fitting ( the acceptable error should be less than 1 )
-
Folder Mode¶
After a folder is selected, and all images in the folder are processed, the program will read summary.csv and rings.csv to produce several maps. You can choose diffrent map display in Representation dropdown menu. The maps will be displayed on the right. To see the image of a particular pixel, you can just simply click on that pixel in the maps. If the image exists, it will be shown on the left. You can change min or max intensity of the image by spin boxes under the image. To see more detail about the image, you can click “More Details” button at the bottom right corner. Then, there is a new window with individual image mode pops up, and you will see all the detail about the image.
Total Intensity Map¶
-
You can change min or max intensity for the maps by spin boxes under the maps.
Ring Intensity Map¶
-
You can change min or max intensity for the maps by spin boxes under the maps.
Orientation and Intensity Vector Field¶
-
You can change the length of vector by dragging the slide bar under the maps.
Elliptical Representation¶
-
Displaying Beam Shape¶
None
- Adjacent images are shown conterminal on the map.
RBF Interpolation
-Alpha Blending
- Beam width and height can be adjusted using the spinboxex.
- Beam size can be either larger or less than step size.
-
If you think there might be some mistakes about fitting results at a pixel, you can fix it by ..
- Click on that pixel
- Click More Details
- Fix it in the individual image mode (You can close the window after fixing)
- Go back to Batch mode window, and click “Refresh Maps”
Image Merger¶
Introduction¶
This program is available on Muscle X version 1.7 or higher
Image Merger is a program which is designed to be used with a series of images with sequential file names taken in a time resolved experiment. It averages a user-selected number of images in an input directory together and saves the new image in an output directory. It does this for the rest of the images in the input directory without user intervention. The user selects an input directory, the number of frames to average together and an output directory.
-
More Details¶
How to use¶
Once the program runs, you will see a new window as below
-
In order to perform auto grouping,
- Select an input directory by pressing “Browse”
- Select an output directory (When an input directory is selected, the program will automatically choose an output directory as “merged_results” folder under input directory. You can change it by pressing “Browse” or simply type the name.
- Select number of frames to average. The program will do auto grouping and display list of groups to Log. Please make sure if images in each group are correct. If number of images can not divided by selected number,
the last group will contain only remaining image(s)
- - Press ‘Start’ to perform average. While the average is being perform, the log section will tell you which group is being averaged, what are images in that group, and what will be the name of result file.
-
Diffraction Centroids¶
Introduction¶
The meridian of the fiber diffraction pattern from muscle consists of families of so-called meridional reflections arising from the myosin packing in the thick filament (43 nm repeat), actin in the thin filaments (2.73 nm inter-subunit repeat), Troponin in thin filaments (42.x nm repeat) and myosin binding protein C (~ 45 nm repeat). Some of these reflections are bunched together in clusters, particularly around the position of the first and second (M1 and M2) myosin meridional reflections. Others are more spread out into well-separated peaks e.g. the 3rd order myosin reflection at 14.3 nm (m3), the 6th order at 7.2 nm (m6), the 15th order myosin reflection at 2.82 nm (m15), the first and second actin meridional reflections at 2.73 (a1) and 1.31 nm (a2) respectively, and the 3rd order Troponin reflection at ~12.8 nm. Small changes in the spacings of the m15 and a1meridional reflections have been used to measure the extensibility of the thick and thin filaments in situ (Huxley et al., 1994). In addition, the relative spacings of the inner part of the 5.9 and 5.1 nm layer lines from actin have been used to measure relative twisting in the thin filaments (Wakabayashi et al., 1994).
Diffraction Centroids is designed to rapidly and accurately measure the spacings of user specified meridional reflections as well as the 5.9 and 5.1 actin layer lines in a series of diffraction images, such as those generated in a time resolved experiment. The peak profiles in integrated intensity traces along the meridian are often not well modeled by standard functions such as gaussians or voigt functions and may change shape during the course of an experiment. Following Huxley et al., 1994, the spacing of the reflection is estimated as the centroid of the top half of the diffraction peak as an unbiased measure with minimal effect of noise. In this process, a peak width and height is estimated allowing calculation of its integrated intensity using a triangular approximation. Once the desired peaks and approximate locations have been specified by the operator, an entire directory of images can be processed without user intervention.
-
More Details¶
How it works¶
This program requires you to specify a range of axial distances containing a meridional or off-meridian peak that you wish to analyze. You can specify as many of these ranges as you wish. If the number of frames to average is more than 1, all selected images will be average. The program will process the averaged image by going through a series of processes in the order given below.
Meridian¶
Required parameters : all peak ranges and names
1. Find Center¶
4. Calculate Integrated Area¶
To calculate the integrated intensity trace, the program will perform similar algorithm for Equator’s box width calculation, but it will sum the intensity along the meridian instead of the equator, so the box will be vertical box. This histogram will be split into 2 histograms (Top and bottom)
5. Calculate start and end point for convex hull¶
The start point for applying convex hull will be R-min while the end point is 0.7*the minimum distance from center to image bound
6. Obtain the Meridional Intensity Histograms¶
This process is also similar to Equator’s get intensity histogram process, but it will integrate the intensity inside the integrated area which is, int his case, a vertical box (along the meridian)
7. Get Peaks¶
Because the peak ranges are provided by the user, in this process, the program will just find the maximum point in each range
8. Calculate Baselines¶
After the peak locations are identified, the program will calculate the baseline as 50% of the height of each peak. You can change these values or percentage manually through the results table in the GUI.
9. Calculate other information¶
Once the peak locations, and baselines have been idnetified, the information listed below will be calculated for each peak.
-
- Width : distance of 2 intersection points between peak(black) and baseline(yellow)at half height
- Centroid : the dot product of x (distance from center) and y (projected intensity) in range of the left and right intersections. Then, divide the result by the sum of y (projected intensity) along the range.
- Area : height * width / (2.35 * 0.3989)
Off-Meridional peaks¶
If off-meridional settings are provided, the program will also go through the processes listed below. Required parameters : x1,x2,x3,x4,peak 59 range, and peak 51 range.
-
10 Get Start and End points for off-meridian Convex hull¶
- Start point : 0.9 * start point of peak 59 range
- End point : 1.1 * end point of peak 51 range
11 Get the Off-Meridian Intensity Histograms¶
There are 4 histograms will be produced from this process, Top-left, Top-right, Bottom-left, and Bottom-right. Left histograms are the vertical integrated intensity between x1 and x2 while the right histograms are vertical integrated intensity between x3 and x4. They are all split to top and bottom by using center y
13 Calculate Baselines for peak 59 and 51¶
Same process as the meridian’s Calculate Baselines
14 Calculate other information for peak 59 and 51¶
Same process as the meridian’s Calculate other information
How to use¶
Once the program is run, you will see the window below
-
There are 2 options for averaging input files, manual-grouping and auto-grouping, For manual-grouping, you can select a single file or multiple files to average. For auto-grouping, you have to select a folder and select the number of frames to average. You can then select the first group to start processing.
-
After selecting input files, you have to add individual meridional peak(s) by inputting its name, and approximate location by range (start and end) in pixels. Then, if you want to get information from the 59 and 51 Angstrom peaks for off-meridian area, you have to check “Off Meridian” and input x1,x2,x3,x4, the approximate location of the 59 and 51 Angstrom peaks as ranges in pixels. x1,x2,x3, and x4 will be the distance from the meridian in pixels while peak 59,51 are the distance from the equator in pixels.
After everything is set and START is pressed, you will see another window pops up as below
-
There are 4 tabs if the Off Meridian is checked. Otherwise, there will be 3 tabs.
Image¶
In this tab, you will see the input image, 2 graphs and a number of options on the right. The 2 graphs are the top and bottom meridional projections. In the option frame on the right, you will see 3 sections, Images, Display Options, and Calculation Settings.
Images¶
In this section, you will see the name of images. In the example above, there is only 1 image selected, so it displaysonly 1 file.
Display Options¶
In this section, you can set what you want to see in the image, set min or max intensities, zoom in and full zoom out. To zoom in the image, you have to click Zoom in Image button and draw a zoom rectangle in the image. You can also see mouse wheeling to zoom in and zoom out.
Calculation Settings¶
In this section, you can do
Before setting manual rotation and center, it’s better to zoom the image to the area of the diffraction because it will be easier to set these parameters correctly. To set the rotation and center, you need to click 2 positions of the image. The first one will be a reflection peak on one side of the meridian, and the second one will be the corresponding (opposite) reflection peak on the other side of the meridian. To cancel, press ESC.
-
This assumes that the center of diffraction is correct. After the button is clicked, the program will allow users to select an angle by moving a line. Clicking on image when the line is on the meridian of the diffraction pattern will set manual rotation angle. To cancel, press ESC.
-
The Meridional Area is the integrated area along the meridian. To set it, you need to click 2 positions in the image which are the left and right side of the box. To cancel, press ESC.
-
Left Off-Meridian area is x1, and x2 while Right Off-Meridian is x3, and x4. These options will be available only when you check Off-Meridian. To set the area, you can do the same thing as Set Meridian Area. To cancel, press ESC.
Top & Bottom Diffraction patterns¶
-
For both Top and Bottom Diffraction tabs, you will see the same GUI providing different information. The graph will be integrated intensity histogram along the meridian. Also, there are 2 sections at the bottom and result table on the right.
Display Options¶
In this section, you can set what you want to see in the graph by checking boxes, and zoom in or full zoom out.
Settings¶
In this section, you can set start and end of convex hull by pressing the button and click start and end location in the graph. After these points are set, the program will apply them to other images in the same folder.
-
Results table¶
This table will display all information for each peaks : name, centroid, baseline, intensity(area).
-
You can reject a peak by checking Reject check box. Also, you can change baseline by just typing the new baseline in the table
-
By default, the baseline will be set to 50% of the peak height. However, you can change it here by typing a new baseline value or percentage by putting ‘%’ at the end.
Off-Meridian¶
This tab will be displayed only when the Off-Meridian option is checked. In this tab, you will see result table, settings section and 4 graphs which are Top-left, Top-right, Bottom-left, and Bottom right integrated intensity inside off-meridian box.
-
The table on the right contains all information for the 59 and 51 Angstrom peaks in each quadrant.
-
You can also change the baseline by typing the new value or percentage (default is 50%)
-
For example, if you set baseline to 80%, the result will be as below
-
Moreover, you can set start and end points for Convex hull background subtraction by pressing Set Convex hull range and click on a graph. This range will apply to all quadrants.
-
DDF Processor¶
Introduction¶
DDF Processor is a program which is able to average data points for ddf file
-
More Details¶
How to use¶
Once the program runs, you will see a new window as below
-
In order to perform generate result file,
- Select an input ddf file by pressing “Browse”. After the file is selected, the program might take few seconds to process the file.
-
- When the file is processed, all available columns will be displayed as check boxes. You can select columns that will be in the result file, and number of point to average.
-
- Press Generate button to generate the result file. After the button is pressed, the save dialog will pop up. You can set location, name of the result file, and output file type (csv, html, or xlsx).
-
Calibration Settings¶
When the calibration image selected, the program will try to fit a circle to the image. The center and radius will be shown above the image if the circle can be fitted.
Calibration Settings
However, if the circle cannot be fit or the circle is in the wrong position, you can also fit the circle manually by clicking on “Set manual calibration by point selections” button. After the button clicked you will see another image at the bottom left which is the zoom area of your cursor.
manual calibration
You will have to click at least 5 points on the ring position, and click Done when you finish. After setting appropriate Silver Behenate, and click OK, the image will be reprocess with a new calibration settings including center and d-spacing.
Also, you can manually set the calibration parameters which are λ, Sdd , and pixel size or fix the center location by input coordinates x,y.
These parameters are used to calculate d10 by 
d10
Image Processing Functions¶
Finding Center¶
To find the center, the image will be converted to an 8 bit image, Gaussian filter is applied to reduce noise, apply thresholding and find the circle by fitting an ellipse to the biggest contour of the threshold image. The center is the center of ellipse.
- 
-
If the circle cannot be determined this way, the center will be calculated by using the moments method of OpenCV to find the center of the object.
Calculate Rotation Angle¶
The program will first calculate the initial angle by fitting an ellipse to the pattern. However, another approach will be applied to find more accurate angle by using azimuthal integration histogram. To find the angle from azimuthal integration histogram is just getting the angle with highest intensity. If the angle from this method is not too different from the initial angle, the angle from this method will be returned. Otherwise, the initial angle will be returned.
-
Following methods are all based on the azimuthal integration described above.
Gaussian Mixture Model (GMM)¶
See Scanning Diffraction for details.
Herman Orientation Factor¶
-
After getting the circular histogram from azimuthal integration, we can calculate Herman Orientation Factor for each degree. The formula is
\(HoF = \frac{3\langle\cos^2\phi\rangle-1}{2}\), where \(\langle\cos^2\phi\rangle = \frac{\sum_{i=0}^{i=x}I_i\cos^2\phi_i\sin\phi_i}{\sum_{i=0}^{i=x}I_i\sin\phi_i}\)
x is either equal to 180 or 90 degrees.
-
We can get the same number of Herman Factors as the length of the histogram. (See Page 7 for properties of Herman Factor.) Following two figures seperately show results of integration on 90 and 180 degrees.
-
-
Note
In Scanning Diffraction program, the area of azimuthal integration is set by ROI. In other programs, the area of azimuthal integration is fixed where the inner radius is 0 and the outer radius is 1/3 max radial length.
Calculate R-min¶
R-min is the minimum radius from the center to be analyzed in a given diffraction pattern. The program will create a radial histogram. The x-axis is radius, and the y-axis is the intensity. R-min is determined as the 50% of its maximum value. However, this radius should not be more than 150% of the maximum point location to prevent over removal. For example, if the maximum point is at 35, R-min should be between 36 - 59. If the program cannot find the 50% of the maximum value in this range, R-min will be 59.
-
Release Notes¶
Version-1.1¶
Release Date : Oct 4, 2017
Updates¶
1. Bio-Muscle¶
- Minor Bug Fixes : Zoom in function didn’t work properly for images from Pilatus sensor
2. Quadrant Folding¶
- Create “bg” folder under “qf_results” and contains background images
- Improve Angular, Circular symmetric, and 2D convex hull background subtraction (smooth)
3. Layer Line Traces¶
- Introducing a new program called Layer Line Traces. To run the program simply run musclex with the shortcut “ll”
Version-1.2¶
Release Date : Oct 11, 2017
Updates¶
1. Layer Line Traces¶
- Add “Process Current Folder” Button
- Change summary.csv file format
- Minor bug fixes : centroids were not displayed properly
2. Quadrant Folding¶
- Add “Process Current Folder” to result tab
- Display current settings when “Process current folder” is pressed
- Modify GUI for background subtraction section (Change checkbox to drop-down list)
- Mask Threshold, Ignore Quadrant, R-min and R-max will be remembered and used for other images in the same folder
- Display “How to” in status bar when a setting button is pressed (e.g. set rotation angle)
- Import Circularly Symmetric Background Subtraction from FibreFix CCP13
- Remove Angular Background Subtraction Method (temporary)
Version-1.3a¶
Release Date : Oct 17, 2017
Updates :¶
Projection Traces (Layer Line Traces)¶
- Change name from Layer Line Traces to Projection Traces
- Support vertical boxes
- Display model results in box tab
- Update CSV format
- Display cross-line when start box selection
- Add ‘Export 1-D Projection’ Check box
- Add colors for background plot
- More details : Click Here
Version-1.4¶
Release Date : Oct 24, 2017
Updates¶
1. Diffraction Centroids¶
- Convex hull range will be remember once it’s set. This range will apply to the next image in the same folder
- Minor Bug Fixes : Zoom in function didn’t work properly when the image is changed
2. Circular Projection¶
- Fixed : application was crashed in some folder
- Fixed : locations in maps were incorrect when number of images are more than 9999 images
- Fixed : error when a pixel on the maps is clicked, but image not found
- Maps will display information from the best ring instead of average ring. These information will be gathered from rings.csv. (The best ring is the ring model that produces lowest fitting error rate and angle sigma is less than 1 (To prevent picking uniform ring)
- Update summary.csv (remove average ring model and add number of rings)
3. Projection Traces¶
- Add Convex hull background subtraction option when a box is added
- Now, you can name the box. You will be asked to put the name of the box and choose background subtraction method when a box is added
- Display convex hull range in each box tab
- Convex hull range can be set manually by pressing “Set Convex Hull Range” and select start and end point in the plot
- Add Zoom-in, Zoom-out, and Full Zoom out function in each box tab
- You can moving around the plot when it’s zoomed by dragging
- Add Distance from center at the status bar when the cursor hovers on the plot
Version-1.4.1¶
Release Date : Oct 24, 2017
Version-1.4.2¶
Release Date : Oct 31, 2017
Updates¶
1. Bio-Muscle¶
- Remove Box width spin box, and remain only check box. The program will remember location of box width in pixel instead of its width.
2. All Packages¶
- Fixed : Error when input image is empty and the ellipse cannot be fit (Ellipse fitting is required for finding rotation angle process)
Version-1.5¶
Release Date : Nov 8, 2017
Updates¶
1. All Packages¶
- Support both PyQt4 and PyQt5 (Python2)
- Fixed: warning from matplotlib due to future deprecation
- Fixed: warning from pandas due to future deprecation
2. Projection Traces¶
- Fixed : Convex hull background area was not shown properly when there’re some minus values
3. Circular Projection¶
- Added : fixed R-min and R-max settings to image tab
- Apply fixed R-min and R-max to the next image when it’s set
- Introducing new feature : Blank image and Mask button which allows user to select black image for background subtraction and generate mask for pyFAI azimuthal integration.
4. Diffraction Centroids¶
- Remember zoom area for top and bottom diffraction when the image is changed (Reverted from 1.4)
- Remember start and end points for convex hull background subtraction for top and bottom diffraction when the image is changed
Version-1.5.1¶
Because PyPI account has been changed, all previous releases have been removed from PyPI server. This version is 1.5.1.0 on PyPI
Release Date : Nov 10, 2017
Version-1.6¶
Release Date : Nov 17, 2017
Updates¶
1. Quadrant Folding¶
- Fixed : Core dump from SIGABRT/SIGSEGV when Circularly Symmetric Background Subtraction is applied, and R-min or R-max is changed to some value
- Added : 2 background subtraction methods : Roving Window and Smoothed (Gaussian/Box car)
- Added : Blank image and mask settings
2. Bio-musclex¶
- Added : Blank image and mask settings
3. Circular Projection¶
- Added : Create a HDF file option for batch mode. When a folder is selected, the program will ask you to select a HDF file, but if you do not have the HDF file, you can create a new one by specify start and point of x and y along with their step size
[[/images/CP/create_HDF.png]]
Version-1.7¶
Release Date : Nov 29, 2017
Updates¶
1. Quadrant Folding¶
- Changed result file name to [input_name]_folded.tif
2. Projection Traces¶
- Fixed : 1-D Histogram export option did not work due to overwritten file name
3. Circular Projection (Batch mode)¶
- Changed UI
- Added : Flip X and Y buttons
- Added : Rings options (Best rings or select ring by distance to center)
- Added : Color map options
Version-1.7.1¶
Release Date : Nov 30, 2017
Updates¶
1. Circular Projection¶
Map¶
- Added new representation : Intensity and rotation maps
- Fixed : total intensity map didn’t show last row and column
- Fixed : incorrect image was displayed when a point on the plot is clicked
- Fixed : distance of ring option was changed and Reload was clicked, but it didn’t take effect
Image mode¶
- Fixed : Calibration Calculation was incorrect
- Fixed : Ignore rings which is further than R-max
Version-1.8¶
Release Date : Dec 5, 2017
Version-1.8.1,1.8.2¶
Release Date : Dec 8, 2017
Version-1.9¶
Release Date : Dec 14, 2017
Version-1.9.1¶
Release Date : Dec 19, 2017
Updates¶
1. Circular Projection¶
- Added : Bandwidth option for mapping. If you select mapping by D-spacing, you have to select distance from center, unit, and bandwidth which is acceptable range from the distance. For example, if the distance is 8.0 and bandwidth is 0.5. All rings at distance between 7.5 to 8.5 will be selected.
-
Project Credits¶
Development is done by current and former students at Illinois Tech with Illinois Tech faculty advising within the BioCAT project. Partial project funding by NIH.
Developers¶
- Jinjian Shao (2015)
- Jiaqi Li (2016)
- Miguel Menendez Alvarez (2016)
- Jiranun Jiratrakanvong (2016, 2017)
- Xintian Li (2018)
- Grant Nikseresht (2019)