For decades the work of Little was the standard when it came to genetics of coat colours in dogs. Observations by breeders suggested that his models needed some modifications and research at the beginning of this century showed that the model that Little created was mostly correct. The description of the various genes and their interaction represents the current knwoledge on this topic.
The dominant allele E allows for the production of eumelanin (black or brown pigment) in the melanocytes (pigment cells) in the skin and hair follicles. If the dog is homozygous for the recessive allele e then the production of eumalnin is inhibited and only the production of phaeomelanin (yellow/red pigment) is left. The MC1R gene encodes for a receptor for Melanocortin which triggers the melanocyte to produce pigment. When there is a double e allele the receptor only triggers the production of phaeomelanin.
The other genes A, K and M have no effect if the dog is homozygous for the recessive e and thus the E locus has a cryptomeric effect on them. This effect is not complete because the colour determined by the other genes is still visible on the nose, paws and eyes.
In some breeds the allele EM (Melanistic mask) is known which produces a black (or brown) face with a yellow/red body. This allele is not found in the Border Collie.
E : the colour determined by the other genes is visible
e : if homozygous for this allele the effect of the other other genes is not visible
(EM : other breeds: dark face on yellow/red body)
This gene is also called Dominant Black. The dominant allele blocks the effect of the A locus and thus promotes the production of eumelanin (if that is allowed by the genes at the E locus). If the dog is not dominant black it can still be black with the right alleles at the A locus. This locus can also contain the allele that cause the brindle effect. This is a very rare colour in Border Collies.
The protein that is produced by this gene (beta-defensin 103) binds to the MC1R receptor and effectively blocks the working mechanism of the A locus (Candille, et al. 2007).
The possible alleles are (in order of dominance):
KB : dominant black - the effect of the A locus is completely blocked
kbr : this allele with another kbr or ky will produce brindle and the A-locus controls the resulting colour
ky : if homozygous then there is no brindle and the A locus controls the colour
If the dog is has at least an "extension" E-allele and at least a "dominant black" KB-allele ( E/- KB/- ) then the B locus determines the coat colour. There are two different alleles (in order of dominance):
B : black coat
b : brown coat (there are four variants of the b allele known in various breeds, but these all produce a brown coat)
The gene works by slightly changing the process of the generation of eumelanin. The product of the mutated TYRP1 enzyme is brown instead of black.
This gene encodes for Agouti Signal Peptide which influences the change either in time or location of the switch between eumelanin and phaeomelanin production. There are four alleles known for this gene (in order of dominance):
ay : produces fawn or sable coat colour. During the life of a hair the pigment production switches from eumelanin to phaeomelanin; each hair has a dark tip and a light base
aw : this is the wild, agouti or sable wolf colour. The pigment production switches between eumalanin and phaeomelanin, producing a banding pattern on each hair
at : also known as tri-colour. In certain areas of the coat (above the eyes, on the cheeks, on the insides of the hind legs and often at the end of the front legs) the melanocytes produce phaeomelanin instead of eumelanin
a : no switch between eumelanin and phaeomelanin
Oddly enough the aw allele has also been found in Border Collies. The aw allele is the original form. A mutation of two nucleotides resulted in the ay allele. Independently an insertion resulted in the at allele. A mutation of a single nucleotide in the at allele resulted in the a allele.
A mutation of a single nucleotide in the gene for melanophilin produces a diluting effect. The recessive allele causes a defect in the transport of melanosomes (particles with pigment). This leads to large clumps of pigment. From a distance this looks like a paler shade of the colour.
D : Normal (dominant) allele: no dilution
d : Recessive allele causing diluted colours
The Merle locus (PMEL gene or SILV gene) is complex, both in the mechanism and the way of the mutation. The SILV gene codes for a protein that forms fibre-like structures in pigment cells. These help both with maintaining the shape of the cell as with the production of pigment. The mutation that results in the merle pattern inserts a block of DNA. This DNA has no functional meaning but it contains an area which is similar to the area in the original gene where the generation of the protein stops. There is also a region that only contains a repetition of one nucleotide.
The length of that repeating region is different in individual dogs and correlates with the type of merle pattern (Murphy, et al. 2018). Short sequences (25—55 base pairs) are found in "cryptic merles" (has the merle gene, but it doesn't show in the coat), dilute merles had sequences of 66—74 base pairs, typical merles correlated with sequences of 78—86 base pairs and harlequin merles were found to have repeating sequences of 81—105 base pairs.
The cells from which pigment cells (melanocytes) are formed (melanoblasts) travel in the embryo from the neural crest (the part from which the spine area is formed) to the various areas in the skin. Each of those melanoblasts determines the coat colour for an area of the skin. In each cell both normal PMEL and mutated PMEL protein is formed (because the extra DNA has an area where the blue print for the protein is cut off which is similar to the original gene; so either of those areas can be used to cut of the blue print). If the inserted DNA is short chances are pretty high that the original PMEL protein is formed and the longer the inserted DNA the higher the chance that the modified PMEL protein is generated. More broken protein means that cell structure and distribution of pigment is affected.
The sequence of repeated nucleotides is also sensitive to changing. It can easily be shortened or lengthened. This can happen during mitosis or meiosis. So, offspring can have a different merle pattern than the parent. This process can also explain the difference in merle pattern between areas of the body. It's also likely that mosaicism plays a role in the merle pattern of standard and harlequin merles.
Murphy et al. also showed that some dogs had more than one insertion. Some labs can measure the size of the repeated sequence(s); some even quite accurately. This helps in detecting and explaining cryptic merles. Because merle only affects eumelanin it is invisible in homozygous ee-recessive animals. DNA-tests can easily tell you if an ee-red animal is in fact merle.
m : normal, no merle
M : merle (dominant)
Some labs will tell the number of repeated nucleotides like Mnnn/m or even in case of multiple insertions: Mnnn/Mxxx/m.
Litlle described four different alleles at this gene (sp piebald spotting, si irish spotting, sw extreme white and S solid colour). DNA research hasn't found evidence for four alleles. There is a mutation that correlates with piebald spotting, but all tested extreme white dogs also had this allele. A lot has to be researched in this area. The alleles show co-dominant expression, dogs which are homozygous for sp have more white than heterozygous animals.
For now some labs offer tests for the mutation for sp.
S : solid
sp : piebald spotting
Little, Clarence C., The Inheritance of Coat Color in Dogs, Ithaca, New York, Comstock Pub. Associates, 1957.
Candille SI, Kaelin CB, Cattanach BM, et al. A -defensin mutation causes black coat color in domestic dogs. Science. 2007;318(5855):1418-23. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2906624/
Murphy, Sarah C., Evans, Jacquelyn M., Tsai, Kate L. and Clark, Leigh Anne, Length variations within the Merle retrotransposon of canine PMEL: correlating genotype with phenotype, Mobile DNA20189:26, https://doi.org/10.1186/s13100-018-0131-6