The gastrointestinal tract of primitive chordates forms as a simple tube.  Amphioxus possesses a homolog of the vertebrate liver, complete with a portal vein system.


The region of the digestive tract of Amphioxus homologous to the vertebrate liver is depicted below.
Hagfish (in the image below), the most primitive craniates, lack jaws and a stomach.  The same situation exists in lampreys.
All other vertebrates possess jaws.  During embryonic development, the first pharyngeal arch forms the maxillary and mandibular processes which will form the upper and lower jaws.  Below is the mouth in a pig embryo.
pig mouth
In non-mammalian vertebrates, the nasal cavity can open directly into the oral cavity, as in the turtle below.
turtle skull
In mammals, this condition was modified so that mammals could continue to breathe while eating.  Maxillary and palatine bones on the lateral sides of the mouth fused in the midline to create a secondary palate which separates the oral and nasal cavities.  These bones fuse with the premaxillary bones which hold the incisors.  Although human adults do not possess premaxillary bones, they exist in fetuses.
fetal premaxillary
Unfortunately, the fusion of these bones may not occur and several variations of a cleft palate can occur.
cleft palate cleft palate
The first vertebrates lacked a stomach.  The first stomachs developed as specialized regions of intestines.  In the human embryo, the stomach begins its development as a tubular region of the gastrointestinal tract which begins as a generalized tube which then widens and rotates (Sadler, p. 241).  Abnormal rotation of the gut can cause complications in newborns and infants, and rarely can appear in adults as well (Konings-Beetstra, 1990). stomach
     Originally the pancreas is composed of separate dorsal and ventral structures.  When the two duct systems fuse, the dorsal duct may be lost or retained as the accessory duct of the pancreas (Sadler, p. 246-7).  In some individuals, additional pancreatic tissue (heterotopic pancreatic tissue) can be found anywhere from the esophagus to the intestines (Sadler, p. 247).  Some people have a bifid gall bladder (Sadler, p. 245).  The repression of the gene Sonic hedgehog in a restricted region of the posterior foregut causes the pancreas to develop.  Expansion of the area in which Shh is repressed can cause pancreatic tissue to develop in a larger area, including the stomach and duodenum (Kim, 1998).  The notochord plays a role in the development of the pancreas (Biemar, 2001). Pancreas development requires Sonic Hedgehog, the homeobox gene Ipf1, the bHLH gene Neurogenin3, and the LIM homeodomain homeodomain Isl1. The POU-homeobox gene TCF2 seems to be essential for the first stages of pancreas differentiation and mutant mice lack a pancreas (Haumaitre, 2005).
development of pancreas
    In bony fish and amphibians, the yolk sac is an expansion of the digestive tract which contains yolk.  In amniotes, the enormous amount of yolk present causes the yolk sac to develop differently.  The majority of the yolk sac is extraembryonic lying outside the embryonic body and composing no portion of the adult digestive system.  One effect of this change in organization is that the intestines are drawn outside of the embryonic body (they are herniated).  In the third month of development the herniated intestines normally return to the body (Sadler, p. 250).  Although human embryos do not require a yolk sac for nutrition (given the existence of the placenta), a yolk sac forms nonetheless and the herniation of the intestines can persist as a serious birth defect.  In 1 in 5,000 births the intestines are herniated while in 1 in 10,000 births, the liver and intestines are herniated (Moore, p.  288).  In the 5th embryonic week the midgut herniates out of the body only to return in the next several weeks, rotating about 270 degrees.  In some cases, the rotation does not reach this degree, leading to abnormalities and potential complications (Konings-Beetstra, 1990).  As the intestines return to the body, they can adhere to other structures, as can the cecum (Moore, p. 290).
intestine herniation intestine herniation
intestine herniation in cat intestine herniation in cat
Some people have duplicated portions of their intestines.  In one case there was a duplication of the rectum which contained gastric tissue (Sadler, p. 254).  Many parts of the gastrointestinal tract can be duplicated, especially the ileum.  The duplicated portion may or may not join with the lumen of the GI tract (Burnett, 1953).
In most vertebrates (and even some primitive mammals), the digestive, urinary, and reproductive tracts terminate in a single opening called the cloaca.  Although these tracts are completely distinct in human adults, human embryos form a cloaca which receives these other tracts.  In some infants, the cloaca persists.  In some, there is an abnormal division of the cloaca and the anus may empty into the urethra.  Most anorectal abnormalities result from an unequal division of the embryonic cloaca (Moore, p.  299-300).  The colon may have a double termination and release fecal matter through the vagina (Chaterjee, 1969).
cloaca fistula

The veriform appendix decreases in size after birth (Moore, p.  288).

The following images depict developing regions of the pig digestive system.

liver intestine