Noise removal for PCA - Revision history

Mjeppesen at 06:22, 20 January 2022

2022-01-20T06:22:47Z

← Older revision		Revision as of 06:22, 20 January 2022
Line 1:		Line 1:
	~~[[Category:Protocols]]~~
	~~[[Category:Metabolomics]]~~
	[[category:Data_Processing_and_Analysis]]		[[category:Data_Processing_and_Analysis]]

Line 40:		Line 39:
	If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.		If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.

	~~See also [[Noise_Removal_for_PCA \| MVAPACK~~ Noise Removal Script]]
			==Noise Removal Script==
			idx = findnearest (x, a)
			[Xrm, abrm] = rmnoise (X, ab, idx)

			Note: Reference the MVAPACK manual for a detailed description of noise removal.

Mjeppesen at 21:49, 13 November 2021

2021-11-13T21:49:37Z

← Older revision		Revision as of 21:49, 13 November 2021
Line 39:		Line 39:
	==Noise cutoff application==		==Noise cutoff application==
	If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.		If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.

	See also [[Noise_Removal_for_PCA \| MVAPACK Noise Removal Script]]		See also [[Noise_Removal_for_PCA \| MVAPACK Noise Removal Script]]

Mjeppesen at 21:49, 13 November 2021

2021-11-13T21:49:16Z

← Older revision		Revision as of 21:49, 13 November 2021
Line 39:		Line 39:
	==Noise cutoff application==		==Noise cutoff application==
	If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.		If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.
			See also [[Noise_Removal_for_PCA \| MVAPACK Noise Removal Script]]

Mjeppesen: Added to Category

2021-09-11T05:03:42Z

Added to Category

← Older revision		Revision as of 05:03, 11 September 2021
Line 1:		Line 1:
	[[Category:Protocols]]		[[Category:Protocols]]
	[[Category:Metabolomics]]		[[Category:Metabolomics]]
			[[category:Data_Processing_and_Analysis]]

	==Prepare the data set==		==Prepare the data set==

MediaWiki default at 02:23, 24 May 2013

2013-05-24T02:23:12Z

← Older revision		Revision as of 02:23, 24 May 2013
Line 11:		Line 11:

	==Z score transformation==		==Z score transformation==
	Z score is used for normalizing the individual spectrum. The scaling of the data set across all the ~~spectra~~ is performed in SIMCA-P+. (UV scaling is by default)		Z score is used for normalizing the individual spectrum:

			<math>Z=\frac{x_i-\overline x}{\sigma}</math>

			The scaling of the data set across all the variables is performed in SIMCA-P+. (UV scaling is by default)

	To enter into excel:		To enter into excel:
Line 22:		Line 26:
	# Add dollar signs after letter in standard deviation equation point and average equation point (Ex: C$480)		# Add dollar signs after letter in standard deviation equation point and average equation point (Ex: C$480)
	# Hit enter, click and drag columns.		# Hit enter, click and drag columns.

	~~<math>Z=\frac{x_i-\overline x}{\sigma}</math>~~

	==Noise cutoff calculation==		==Noise cutoff calculation==

MediaWiki default at 02:22, 24 May 2013

2013-05-24T02:22:36Z

← Older revision		Revision as of 02:22, 24 May 2013
Line 3:		Line 3:

	==Prepare the data set==		==Prepare the data set==
	1. The data can be prepared in txt file from ACDLab 1D processor.		# The data can be prepared in txt file from ACDLab 1D processor.
			# After the spectra are "Autophased" and "Referenced" to TMSP correctly, click the "Integration" icon in the tool bar.
	2. After the spectra are "Autophased" and "Referenced" to TMSP correctly, click the "Integration" icon in the tool bar.		# Click the "Series" from the menu and choose "Table of common integrals".
			# Then export table to the targeted file folder.
	3. Click the "Series" from the menu and choose "Table of common integrals".		# Open the file in Office Excel. Delete the first row and insert a new row below the row of sample numbers.
			# Fill the row with sample class names.
	4. Then export table to the targeted file folder.

	5. Open the file in Office Excel. Delete the first row and insert a new row below the row of sample numbers.

	6. Fill the row with sample class names.

	==Z score transformation==		==Z score transformation==

MediaWiki default at 02:22, 24 May 2013

2013-05-24T02:22:14Z

← Older revision		Revision as of 02:22, 24 May 2013
Line 19:		Line 19:

	To enter into excel:		To enter into excel:
	1) Click the first row, first column of data		# Click the first row, first column of data
	2) Add in minus sign		# Add in minus sign
	3) Click first average data point		# Click first average data point
	4) Put () around first 2 terms in equation		# Put () around first 2 terms in equation
	5) Add in division sign		# Add in division sign
	6) Click first standard deviation data point		# Click first standard deviation data point
	7) Add dollar signs after letter in standard deviation equation point and average equation point (Ex: C$480)		# Add dollar signs after letter in standard deviation equation point and average equation point (Ex: C$480)
	8) Hit enter, click and drag columns.		# Hit enter, click and drag columns.

			<math>Z=\frac{x_i-\overline x}{\sigma}</math>
	<math>Z=(x_i-\overline x)/\sigma</math>

	==Noise cutoff calculation==		==Noise cutoff calculation==

	This is based on the Excel template that is exported directly from the ACDLabs.		This is based on the Excel template that is exported directly from the ACDLabs. The calculation is based on the z-score data set.

	0. The calculation is based on the z-score data set.

	~~1. Cross the board, calculate the standard deviation and average values for each row.~~

	~~2. Calculate the absolute value for relative standard deviation by dividing the standard deviation by the absolute average values.~~

	3. Find out the average value and standard deviation for the pre-assigned noise region of bins for each sample (chemical shift<0ppm or >10ppm). Calculate the cutoff equals to the average plus 3 times standard deviation.

	4. Only when the z score is smaller than 0, AND the value of relative standard deviation is smaller than the cutoff of the noise, then that bin can be considered as a noise bin. All noise region-defined bins should be set to 0 and remove from the analysis data sets.		# Across the board, calculate the standard deviation and average values for each row.
			# Calculate the absolute value for relative standard deviation by dividing the standard deviation by the absolute average values.
			# Find out the average value and standard deviation for the pre-assigned noise region of bins for each sample (chemical shift<0ppm or >10ppm). Calculate the cutoff equals to the average plus 3 times standard deviation.
			# Only when the z score is smaller than 0, AND the value of relative standard deviation is smaller than the cutoff of the noise, then that bin can be considered as a noise bin. All noise region-defined bins should be set to 0 and remove from the analysis data sets.

	==Noise cutoff application==		==Noise cutoff application==
	If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.		If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.

Bozhang: /* Noise cutoff calculation */

2013-05-17T02:44:01Z

Noise cutoff calculation

← Older revision		Revision as of 02:44, 17 May 2013
Line 32:		Line 32:

	==Noise cutoff calculation==		==Noise cutoff calculation==

			This is based on the Excel template that is exported directly from the ACDLabs.

	0. The calculation is based on the z-score data set.		0. The calculation is based on the z-score data set.

	1. ~~For each class~~, calculate the standard deviation and average values.		1. Cross the board, calculate the standard deviation and average values for each row.

	2. Calculate the absolute value for relative standard deviation by dividing the standard deviation by the absolute average values.		2. Calculate the absolute value for relative standard deviation by dividing the standard deviation by the absolute average values.

	3. Find out the ~~maximum~~ for each ~~row. If the maximum is smaller than 0. It indicates all the z score values are smaller than 0.~~		3. Find out the average value and standard deviation for the pre-assigned noise region of bins for each sample (chemical shift<0ppm or >10ppm). Calculate the cutoff equals to the average plus 3 times standard deviation.

	4. ~~Find out the region that no peak exists, find out~~ the ~~maximum of~~ the ~~relative~~ standard deviation ~~for each class~~.

	5. Only when the z score is smaller than 0, AND the value of relative standard deviation is smaller than the ~~maximum~~ of the noise ~~region~~, then that bin can be considered as a noise ~~region~~. All noise region-defined bins should be set to 0.		4. Only when the z score is smaller than 0, AND the value of relative standard deviation is smaller than the cutoff of the noise, then that bin can be considered as a noise bin. All noise region-defined bins should be set to 0 and remove from the analysis data sets.

	==Noise cutoff application==		==Noise cutoff application==
	If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.		If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.

Bozhang: /* Z score transformation */

2013-03-29T01:21:46Z

Z score transformation

@@ Line 17: / Line 17: @@
 ==Z score transformation==
 Z score is used for normalizing the individual spectrum. The scaling of the data set across all the spectra is performed in SIMCA-P+. (UV scaling is by default)
 <math>Z=(x_i-\overline x)/\sigma</math>
 ==Noise cutoff calculation==

Bozhang: /* Noise cutoff calculation */

2012-12-23T03:08:09Z

Noise cutoff calculation

← Older revision		Revision as of 03:08, 23 December 2012
Line 31:		Line 31:
	4. Find out the region that no peak exists, find out the maximum of the relative standard deviation for each class.		4. Find out the region that no peak exists, find out the maximum of the relative standard deviation for each class.

	5. Only when the z score is smaller than 0, AND the value of relative standard deviation is smaller than the maximum of the noise region, then that bin can be considered as a noise region.		5. Only when the z score is smaller than 0, AND the value of relative standard deviation is smaller than the maximum of the noise region, then that bin can be considered as a noise region. All noise region-defined bins should be set to 0.


	==Noise cutoff application==		==Noise cutoff application==
	If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.		If the data set is prepared for PCA, only the noise region across the whole data set can be removed. For data set for OPLS-DA, the noise region determined for each class can be removed separately.