Sørensen–Dice coefficient

The Sørensen–Dice index, also known by other names (see Names, below), is a statistic used for comparing the similarity of two samples. It was independently developed by the botanists Thorvald Sørensen and Lee Raymond Dice, who published in 1948 and 1945 respectively.

Name

The index is known by several other names, usually Sørensen index or Dice's coefficient. Both names also see "similarity coefficient", "index", and other such variations. Common alternate spellings for Sørensen are Sorenson, Soerenson index and Sörenson index, and all three can also be seen with the –sen ending.

Other names include:

• Czekanowski's binary (non-quantitative) index

Quantitative version

The expression is easily extended to abundance instead of presence/absence of species. This quantitative version is known by several names:

• Quantitative Sørensen–Dice index
• Quantitative Sørensen index
• Quantitative Dice index
• Bray-Curtis similarity (1 minus the Bray-Curtis dissimilarity)
• Czekanowski's quantitative index
• Steinhaus index
• Pielou's percentage similarity
• 1 minus the Hellinger distance

Formula

Sørensen's original formula was intended to be applied to presence/absence data, and is

$QS = \frac{2C}{A + B} = \frac{2 |A \cap B|}{|A| + |B|}$

where A and B are the number of species in samples A and B, respectively, and C is the number of species shared by the two samples; QS is the quotient of similarity and ranges between 0 and 1.

It can be viewed as a similarity measure over sets:

$s = \frac{2 | X \cap Y |}{| X | + | Y |}$

Similarly to Jaccard, the set operations can be expressed in terms of vector operations over binary vectors A and B:

$s_v = \frac{2 | A \cdot B |}{| A |^2 + | B |^2}$

which gives the same outcome over binary vectors and also gives a more general similarity metric over vectors in general terms.

For sets X and Y of keywords used in information retrieval, the coefficient may be defined as twice the shared information (intersection) over the sum of cardinalities :

When taken as a string similarity measure, the coefficient may be calculated for two strings, x and y using bigrams as follows:

$s = \frac{2 n_t}{n_x + n_y}$

where n_t is the number of character bigrams found in both strings, n_x is the number of bigrams in string x and n_y is the number of bigrams in string y. For example, to calculate the similarity between:

night

nacht

We would find the set of bigrams in each word:

{ni,ig,gh,ht}

{na,ac,ch,ht}

Each set has four elements, and the intersection of these two sets has only one element: ht.

Inserting these numbers into the formula, we calculate, s = (2 · 1) / (4 + 4) = 0.25.

Difference from Jaccard

This coefficient is not very different in form from the Jaccard index. However, since it doesn't satisfy the triangle inequality, it can be considered a semimetric version of the Jaccard index.

The function ranges between zero and one, like Jaccard. Unlike Jaccard, the corresponding difference function

$d = 1 - \frac{2 | X \cap Y |}{| X | + | Y |}$

is not a proper distance metric as it does not possess the property of triangle inequality. The simplest counterexample of this is given by the three sets {a}, {b}, and {a,b}, the distance between the first two being 1, and the difference between the third and each of the others being one-third. To satisfy the triangle inequality, the sum of any two of these three sides must be greater than or equal to the remaining side. However, the distance between {a} and {a,b} plus the distance between {b} and {a,b} equals 2/3 and is therefore less than the distance between {a} and {b} which is 1.

Applications

The Sørensen–Dice coefficient is mainly useful for ecological community data (e.g. Looman & Campbell, 1960). Justification for its use is primarily empirical rather than theoretical (although it can be justified theoretically as the intersection of two fuzzy sets). As compared to Euclidean distance, Sørensen distance retains sensitivity in more heterogeneous data sets and gives less weight to outliers. Recently the Dice score (and its variations, e.g. logDice taking a logarithm of it) has become popular in computer lexicography for measuring the lexical association score of two given words.

See also

• Correlation
• Czekanowski similarity index
• Jaccard index
• Hamming distance
• Horn’s index
• Hurlbert’s index
• Kulczyński similarity index
• Pianka's index
• MacArthur and Levin's index
• Mantel test
• Morisita's overlap index
• Most frequent k characters
• Overlap coefficient
• Renkonen similarity index (due to Olavi Renkonen)
• Simplified Morisita’s index
• Tversky index
• Universal adaptive strategy theory (UAST)

External links

• Open Source Dice / Sorensen Scala implementation as part of the larger stringmetric project

• Gallagher, E.D., 1999. COMPAH Documentation, University of Massachusetts, Boston
• http://www.sekj.org/PDF/anbf40/anbf40-415.pdf
• Looman, J. and Campbell, J.B. (1960) Adaptation of Sorensen's K (1948) for estimating unit affinities in prairie vegetation. Ecology 41 (3): 409–416.
• Roberts, D.W. (1986) Ordination on the basis of fuzzy set theory. Vegetatio 66 (3): 123–131.
• McCune, Bruce & Grace, James (2002) Analysis of Ecological Communities. Mjm Software Design; ISBN 0-9721290-0-6.
• Rychlý, P. (2008) A lexicographer-friendly association score. Proceedings of the Second Workshop on Recent Advances in Slavonic Natural Language Processing RASLAN 2008: 6–9