Richard Burton
Richard Walter Jenkins
Richard Burton (actor)

Prompt:
Answer the question based on the given passage. Only give me the answer and do not output any other words. The following are some examples.

Passage:
Harry Hole
Harry Hole is the main character in a series of crime novels written by Norwegian author Jo Nesbø. Hole is a brilliant and driven detective with unorthodox methods, a classic loose cannon in the police force.

Character

Harry Hole, a police officer with the Oslo Crime Squad, lost his mother, a descendant of the Sami people, to cancer while he was in his twenties, and he has never had a close relationship to his father, Olav Hole, a former teacher. He was born in 1965, and has a younger sister with Down syndrome. He is unmarried, but has had close relationships with a number of women throughout the series of ten novels. Otherwise, he has few close friends – and the ability to make enemies from amongst his colleagues who, nevertheless, grudgingly respect him.

Hole is a heavy smoker and an alcoholic although, at times, his alcoholism is under control. The effects of his problem sometimes bring him into repeated conflict with his superiors and some colleagues, but Bjarne Møller, head of Hole's department and one of his closest friends, manages to prevent him from being sacked as a result, primarily because he recognises that Harry Hole is a brilliant detective who is recognised amongst his colleagues because of his role in the solving the case of a serial killer in Australia. Hole is one of few in the force who has undergone special training in police interrogation techniques and firearms by the FBI.

Hole has few friends within the Oslo Police Department, exceptions being some staff at the forensics division of the Norwegian police, including Beate Lønn, whom he often uses to secure crucial information in critical situations where specialist knowledge is required, and Bjørn Holm, another skilled forensics officer. Harry is also friendly with Gunnar Hagen, his former senior officer prior to the case chronicled in Phantom.

In many ways, the author's home city of Oslo has the starring role in the Harry Hole novels. Much of the background detail of the stories involves real locations, and the city is shown "warts and all", ranging from Hole's favourite eating and drinking places that he frequents, to the traffic of the so-called "Traffic Machine" road system, the real-life headquarters of the Police Department, the swimming pool at Frogner Park, to the haunts of neo-Nazis, drug addicts and prostitutes, all of this described vividly (in the various novels) at different times of the year – in the snow and freezing weather of winter to the high temperatures of a particularly warm summer. Harry Hole's friends and acquaintances include city residents from every social background, including immigrants from other parts of the world to old school friends such as Øystein Eikeland, the taxi driver – possibly the person with whom he is closest.

Harry Hole's home address is in Sofies Gate in Bislett, Oslo. In Phantom he no longer lives here, as he has been in Hong Kong for three years. Near his place of residence is his favourite "watering hole", Restaurant Schrøder (Schrøder's, for short) in St. Hanshaugen, which figures in most of the novels. He has had a serious relationship with Rakel Fauke, whose son, Oleg, looks up to Harry as a father figure. After the case chronicled in The Snowman, their relationship is put under severe pressure.

"Hole", Harry's family name, is the name of a historic Norwegian town (Hole, Norway), with a heritage that goes back to Norway's origins in the Viking Age. The name is derived from Old Norse Hólar, the plural form of hóll, meaning "round (and isolated) hill". The word is pronounced as two syllables, with stress on the first (hoh-leh), as is implied in The Bat, where the Australian police call him "Harry Holy".

Appearances in the Harry Hole series

* In The Bat (1997), Hole is sent to Sydney, Australia to aid the Australian police in their investigation of the murder of a Norwegian citizen.
* In Cockroaches (1998), Hole is sent to Thailand to investigate the murder of the Norwegian ambassador.
* In The Redbreast (2000), Hole tracks an assassin planning an attack on a prominent member of the establishment.
* In Nemesis (2002), Hole investigates a fatal bank robbery and becomes implicated in the apparent murder of an ex-girlfriend.
* In The Devil's Star (2003), Hole investigates a series of serial killings and suspects a fellow policeman of criminal activity.
* In The Redeemer (2005), Hole is on the trail of a Croatian hitman who kills a Salvation Army officer during a Christmas street concert.
* In The Snowman (2007), Hole struggles to identify Norway's first serial killer.
* In The Leopard (2009), Hole returns from self-imposed exile in Hong Kong and unofficially investigates a serial killer.
* In Phantom (2011), Hole again returns from Hong Kong to look into a murder apparently committed by his would-be son, Oleg. His investigation draws him into Oslo's drug scene.
* In Police (2013), Hole is needed once again to discover the identity of a serial killer who is stalking the streets of Oslo, killing police officers who previously investigated old cases.

Books

*1997 – Flaggermusmannen; English title: The Bat (2012)
*1998 – Kakerlakkene; English title: Cockroaches (2013)
*2000 – Rødstrupe; English translation: The Redbreast (2006)
*2002 – Sorgenfri; English translation: Nemesis (2008)
*2003 – Marekors; English translation: The Devil's Star (2005)
*2005 – Frelseren; English translation by Don Bartlett: The Redeemer (2009)
*2007 – Snømannen; English translation by Don Bartlett: The Snowman (2010)
*2009 – Panserhjerte; English translation by Don Bartlett: The Leopard (2011)
*2011 – Gjenferd; English translation by Don Bartlett: Phantom (2012)
*2013 - Politi; English translation: Police (2013)
Question:
Which author created the detective Harry Hole?
Answer:
Jo Nesbø
Passage:
Lake Constance
Lake Constance (German: Bodensee) is a lake on the Rhine at the northern foot of the Alps, and consists of three bodies of water: the Obersee ("upper lake"), the Untersee ("lower lake"), and a connecting stretch of the Rhine, called the Seerhein.

The lake is situated in Germany, Switzerland and Austria near the Alps. Its shorelines lie in the German states of Bavaria and Baden-Württemberg, the Austrian state of Vorarlberg, and the Swiss cantons of Thurgau, St Gallen and Schaffhausen. The river Rhine flows into the lake from the south, with its original course forming the Austro-Swiss border, and has its outflow of the "Lower Lake" where it to some extent forms the German-Swiss border until the city Basel.

Description

Freshwater Lake Constance is central Europe's third largest, after Lake Balaton and Lake Geneva. It is 63 km long, and at its widest point, nearly 14 km. It covers approximately 571 km2,  and is 395 m above sea level.  The greatest depth is 252 m in the middle of the eastern part (Obersee). Its volume is approximately 10 e9m3. The lake has four parts: the main section, called Obersee, 476 km2; the north section, Überlinger See, 61 km2; the west section, Untersee, 63 km2; and the northwest section, the Zeller See and Gnadensee. The regulated Rhine flows into the lake in the southeast, through the Obersee, the city of Konstanz and the Untersee, and flows out near Stein am Rhein. Lake Constance provides fresh water to many cities in south Germany.  The culminating point of the lake's drainage basin is the Tödi at 3,614 metres above sea level.

The lake itself is an important drinking water source for southwestern Germany, called Bodensee-Wasserversorgung ("Lake Constance Water Supply").  Car ferries link Romanshorn to Friedrichshafen, and Konstanz to Meersburg.

History

Lake Constance was formed by the Rhine Glacier during the ice age and is a zungenbecken lake. The Rhine, the Bregenzer Ache, and the Dornbirner Ache carry sediments from the Alps to the lake, thus gradually decreasing the depth and coastline extension of the lake in the southeast.

Lake Constance was first mentioned by the Roman geographer Pomponius Mela about 43 AD. He noted that the Rhine flows through two lakes, and gave them the Latin names Lacus Venetus (Obersee) and Lacus Acronius (Untersee). Pliny the Elder used the name Lacus Brigantinus, after the Roman city of Brigantium (today Bregenz). The lake is also colloquially known as the Swabian Sea  (das schwäbische Meer).

The lake was frozen in the years 1077 (?), 1326 (partial), 1378 (partial), 1435, 1465 (partial), 1477 (partial), 1491 (partial?), 1517 (partial), 1571 (partial), 1573, 1600 (partial), 1684, 1695, 1709 (partial), 1795, 1830, 1880 (partial), and 1963.

Approximately 1000 t of fish were caught by 150 professional fishermen in 2001 which was below the previous ten year average of 1200 t per year. The Lake Constance trout (Salmo trutta) was almost extinct in the 1980s due to pollution, but thanks to protective measures they have made a significant return. Lake Constance was the home of the now extinct species of trout Salvelinus profundus,  as well as of the Lake Constance whitefish (Coregonus gutturosus).

Name

After the Council of Constance, the Latin-speaking Catholic world gave the lake its current international name. It was derived from the city of Konstanz, that, in turn, was named after a Roman emperor (either Constantius Chlorus or his grandson Constantius II). The German name, Bodensee, derives from the town of Bodman, situated at a nearby branch of the lake some 8 km northwest of Konstanz.

International borders

Lake Constance is the only area in Europe where no borders exist, because there is no legally binding agreement as to where the borders lie between Switzerland, Germany and Austria.   However, Switzerland holds the view that the border runs through the middle of the lake, Austria is of the opinion that the contentious area belongs to all the states on its banks, and Germany holds an ambiguous opinion. Legal questions pertaining to ship transport and fishing are regulated in separate treaties.

Naturally, disputes arise. One concerns a houseboat which was moored in two states (ECJ c. 224/97 Erich Ciola); another concerns the rights to fish in the Bay of Bregenz. In relation to the latter, an Austrian family was of the opinion that it alone had the right to fish in broad portions of the bay. However, this was accepted neither by the Austrian courts nor by the organs and courts of the other states.

Recent floods

* A 100-year flood around June 1999 (Pfingsthochwasser 1999) raised the level about 2 metres above normal, flooding harbors and many shoreline buildings and hotels.
* In late August 2005, heavy rain raised the level by more than 70 cm in a few days.  The rains caused widespread flooding and washed out highways and railroads.

Islands in the lake

The three major islands are:
*Mainau Island
*Reichenau Island
*Lindau

These are all of the islands and former islands in Lake Constance, listed from east to west:

Towns and cities at the lake

Austria

*Hard
*Hörbranz
*Bregenz
*Lochau
*Höchst
*Fussach
*Gaissau

Germany

From the entry of the Rhine, on the northern or right shore:
*On the Upper Lake (Obersee) and Überlinger See
**Lindau (in Bavaria)
**Wasserburg
**Nonnenhorn
**Kressbronn (in Baden-Württemberg)
**Langenargen
**Eriskirch
**Friedrichshafen
**Immenstaad
**Hagnau
**Stetten
**Meersburg
**Uhldingen-Mühlhofen (on Überlinger See)
**Überlingen
**Sipplingen
**Bodman-Ludwigshafen
*Konstanz with suburbs
**Wallhausen
**Dingelsdorf
**Litzelstetten
*On the lower lake (Untersee)
**Reichenau (including the island with same name)
**Allensbach (on Gnadensee)
**Radolfzell (on Zellersee)
**Moos
**Gaienhofen
**Öhningen

Switzerland

From the entry of the Rhine, on the southern or left shore:
*On the Upper Lake (Obersee)
**Altenrhein, St. Gallen
**Rorschach, St. Gallen
**Horn, Thurgau
**Steinach, St. Gallen
**Arbon, Thurgau (as all the following)
**Frasnacht
**Egnach
**Romanshorn
**Uttwil
**Kesswil
**Güttingen
**Altnau
**Landschlacht
**Münsterlingen
**Bottighofen
**Kreuzlingen (and Konstanz, Germany)
*On the Rhine
**Gottlieben
*On the Lower Lake (Untersee)
**Ermatingen
**Mannenbach
**Berlingen
**Steckborn
**Mammern
**Eschenz
**Stein am Rhein, Schaffhausen
Question:
What is the German name for Lake Constance?
Answer:
Boden-see
Passage:
SPEEDWAY LEGEND IVAN MAUGER AT OTA THIS ... - Somerset Rebels
Somerset Rebels Speedway Official Website: SPEEDWAY LEGEND IVAN MAUGER AT OTA THIS FRIDAY
SPEEDWAY LEGEND IVAN MAUGER AT OTA THIS FRIDAY
Sunday  July 24, 2011
6-time World Speedway Champion, Ivan Mauger, will be at the Oak Tree Arena this Friday (29th) to promote his autobiography.
Mauger is touring British tracks promoting and selling copies of his autobiography ‘The Will To Win’ which will be on sale prior to Somerset’s Premier League match against Redcar, with Mauger personally signing each copy sold!
‘The name of Ivan Mauger is synonymous with the history of speedway.  Quite simply put, he is the most successful and decorated rider the sport has ever known, acknowledged as such by critical acclaim and voted Man of the Millennium by speedway followers.
From humble beginnings in Christchurch, New Zealand, Mauger ventured overseas as a 17 year-old, enjoyed a stellar career, winning his 15th FIM Gold Medal a month before his 40th birthday.
In the same meticulous style that characterised his professional approach to racing, Mauger now relates his life story with co-author, Martin Rogers.’
Share This:
Question:
Who was voted speedway’s man of the millennium
Answer:
Ivan Mauger
Passage:
So THIS is why Henry VIII called wife No.4 the Flanders ...
So THIS is why Henry VIII called wife No.4 the Flanders Mare! Anne of Cleves is an upside down image of the king in portraits | Daily Mail Online
So THIS is why Henry VIII called wife No.4 the Flanders Mare! Anne of Cleves is an upside down image of the king in portraits
From the look on his face, Henry VIII is certainly upset about something.
Turn the portrait upside down and you will see what’s on his mind.
Hidden in the king’s features is none other than Anne of Cleves – the bride he considered so ugly that he never consummated their marriage.
Upside down: Henry in the inverted picture and Whistler's Anne of Cleves portrait
The double portrait from the 1940s is the work of artist Rex Whistler and is to be revealed on tomorrow’s episode of Antiques Roadshow.
Show expert Rupert Maas said: ‘It’s just great fun. Anne wasn’t a very attractive woman according to popular history – although I’m not sure she was quite as hideous as this.’
Anne was the fourth of Henry’s six wives, sent over from Germany after he saw a flattering portrait by Holbein. On arrival, he called her the ‘Flanders Mare’.
RELATED ARTICLES
Share this article
Share
They divorced six months after marrying in 1540. He married twice more while she remained in England and was given a generous settlement.
Henry and Anne are one of many double-acts Whistler made into reversible portraits. Mr Maas said: ‘Whistler preferred the old masters to modernism, which is why he drew and painted in an old-fashioned style.’
A bit more flattering: The Hans Holbein portraits of the couple from 1539
The artist, who died in 1944, gave the portrait to the current owner when she was seven after she sat for him.
The Roadshow programme from Enniskillen, Northern Ireland will be shown on BBC1 tomorrow at 8pm.
Question:
Which of Henry V111's wives was known as The Flanders Mare
Answer:
Flanders mare
Passage:
Phil Cunningham (folk musician)
Phil Cunningham, MBE, born 1960 in Edinburgh, Scotland is a Scottish folk musician and composer. He is best known for playing the accordion with Silly Wizard, as well as in other bands and in duets with his brother, Johnny. When they played together, Johnny and Phil would often horse around, as only brothers can, egging each other on to play faster and faster, and trying, light-heartedly, to trip each other up.   Phil has played with other great Celtic musicians, such as Aly Bain.

Biography

Phil played accordion and violin from a very young age. He attended school in Portobello, and was raised Mormon, attending church regularly and playing organ.  However, by age fifteen due to issues with the Church of Jesus Christ of Latter-day Saints he left, and now describes himself as a spiritualist.

At the age of 16, he joined his older brother Johnny in the group Silly Wizard, where he played accordion, tin whistle, harmonium, guitar, synthesizer, and he sang.  He also wrote many of the group's songs.  After the breakup of Silly Wizard, Phil and Johnny recorded two albums and toured with Irish siblings Mícheál Ó Domhnaill and Triona Ni Domhnaill as the quartet Relativity.  Phil has since had a successful solo career, releasing the albums Airs & Graces and The Palomino Waltz, and several collaborations with fellow Scottish folk musician Aly Bain. He has produced several albums with American singer Connie Dover.  In recent years, Phil has also composed classical music and music for theater and television, with 1997 seeing the premiere of his Highlands and Islands Suite at the Glasgow Royal Concert Hall.  In 2002, Phil was awarded the MBE for services to Scottish music.  He was awarded an honorary Doctor of Letters at Glasgow Caledonian University's graduation ceremony on 27 November 2007.[http://www.gcal.ac.uk/news/news/graduation/slide3.html Graduation Honorary Awards : News Headlines : Glasgow Caledonian University]

Discography

Solo albums

*Airs & Graces (1984)
*Palomino Waltz (1989)

Silly Wizard

*Caledonia's Hardy Sons (1978)
*So Many Partings (1980)
*Wild and Beautiful (1981)
*Kiss the Tears Away (1983)
*A Glint of Silver (1986)
*Live Wizardry (1988)

Relativity

*Relativity (1986)
*Gathering Pace (1987)

With Johnny Cunningham

*Against the Storm (1980)

With Aly Bain

*The Pearl (1995)
*The Ruby (1997)
*Another Gem (2000)
*Spring The Summer Long (2003)
*Best of Aly and Phil (2004)
*Roads Not Travelled (2006)
*Portrait (2010)
*Five and Twenty (2012)

With Connie Dover

*Somebody (1991)
*The Wishing Well (1994)
*If Ever I Return (1997)
*The Border of Heaven (2000)

With Kris Drever

*Mark the Hard Earth (2010)
Question:
Weird Al Yankovic, Phil Cunningham, Benny Andersson of Abba and Sharon Shannon are all noted exponents of what musical instrument?
Answer:
Fisarmonica
Passage:
Java Sea
The Java Sea () is a large (320,000-km²) shallow sea on the Sunda Shelf formed as sea levels rose at the end of the last ice age.  The Java Sea lies between the Indonesian islands of Borneo to the north, Java to the south, Sumatra to the west, and Sulawesi to the east. Karimata Strait to its northwest links it to the South China Sea.

The sea measures about 1,500 km (900 miles) east-west by 420 km (260 miles) north-south and occupies a total surface area of 433,000 square km (167,000 square miles). It covers the southern section of the 1,790,000-square-km (690,000-square-mile) Sunda Shelf.

A shallow sea, it has a mean depth of 46 metres (151 feet). The almost uniform flatness of the sea bottom and the presence of drainage channels (traceable to the mouths of island rivers) indicate that the Sunda Shelf was once a stable, dry, low-relief land area (peneplain) above which were left standing a few monadnocks (granite hills that by virtue of their resistance to erosion form the present islands).

Extent

The International Hydrographic Organization (IHO) defines the Java Sea as being one of the waters of the East Indian Archipelago. The IHO defines its limits as follows:

On the North. By the Southern limit of the South China Sea [Lucipara Point () thence to Tanjong Nanka, the Southwest extremity of Banka Island, through this island to Tanjong Berikat the Eastern point (), on to Tanjong Djemang () in Billiton, along the North coast of this island to Tanjong Boeroeng Mandi () and thence a line to Tanjong Sambar () the Southwest extreme of Borneo], the South coast of Borneo and the Southern limit of Makassar Strait [By a line from the Southwestern extreme of Celebes (), through the Southern point of Tana Keke, to the Southern extreme of Laoet () thence up the West coast of that island to Tanjong Kiwi and thence across to Tanjong Petang, Borneo () at the Southern end of Laoet Strait].

On the East. By the Western limit of Flores Sea [A line from Tg Sarokaja () to the Western Paternoster island () thence to the Northeastern Postiljon Island () and to the West point of Laikang Bay, Celebes].

On the South. By the Northern and Northwestern limits of Bali Sea [A line from the Western Paternoster Island to the East point of Sepandjang and thence through this island to the West point of Gedeh Bay on the South coast of Kangean (). A line from the West point of Gedeh Bay, Kangean Island, to Tg Sedano, the Northeast extreme of Java and down the East coast to Tg Bantenan, the Southeast extreme of the island], the North and West coasts of Java to Java Hoofd () its Western point, and thence a line to Vlakke Hoek () the Southern extreme of Sumatra.

On the West. The East coast of Sumatra between Vlakke Hoek and Lucipara Point ().

History

The Battle of the Java Sea from February to March 1942, was one of the costliest naval battles of World War II. The naval forces of the Netherlands, Britain, Australia, and the United States were nearly destroyed trying to defend Java from Japanese attack.

On 28 December 2014, Indonesia AirAsia Flight 8501 crashed into the Java Sea while on route to Singapore from Surabaya, Indonesia due to bad weather. All passengers and crew died.

Economic activities

The southern section of the seafloor has long been recognized as geologically similar to northern Java, where oil fields occur and extend under the sea. Prospects are also favourable for oil fields in the waters off southeast Kalimantan. As the site of successful exploration for petroleum and natural gas, the Java Sea has become the basis of Indonesia’s export program.

Fishing is an important economic activity in the Java Sea. Over 3,000 species of marine life are found in the area. A number of national parks exist in the area such as Karimunjawa. The Thousand Islands are located north of the national capital Jakarta, and are the city's only regency.

The area around the Java Sea is also a popular tourist destination. Scuba diving offers a chance to explore and photograph underwater caverns, wrecks, coral, sponges, and other marine life.
Question:
The shallow Java Sea is between the islands of Java and what?
Answer:
Kalimantan Borneo
Passage:
Chalk
Chalk is a soft, white, porous sedimentary carbonate rock, a form of limestone composed of the mineral calcite. Calcite is calcium carbonate or CaCO3. It forms under reasonably deep marine conditions from the gradual accumulation of minute calcite shells (coccoliths) shed from micro-organisms called coccolithophores. Flint (a type of chert unique to chalk) is very common as bands parallel to the bedding or as nodules embedded in chalk. It is probably derived from sponge spicules or other siliceous organisms as water is expelled upwards during compaction. Flint is often deposited around larger fossils such as Echinoidea which may be silicified (i.e. replaced molecule by molecule by flint).

Chalk as seen in Cretaceous deposits of Western Europe is unusual among sedimentary limestones in the thickness of the beds. Most cliffs of chalk have very few obvious bedding planes unlike most thick sequences of limestone such as the Carboniferous Limestone or the Jurassic oolitic limestones. This presumably indicates very stable conditions over tens of millions of years.

Chalk has greater resistance to weathering and slumping than the clays with which it is usually associated, thus forming tall steep cliffs where chalk ridges meet the sea. Chalk hills, known as chalk downland, usually form where bands of chalk reach the surface at an angle, so forming a scarp slope. Because chalk is well jointed it can hold a large volume of ground water, providing a natural reservoir that releases water slowly through dry seasons.

Deposits

The Chalk Group is a European stratigraphic unit deposited during the late Cretaceous Period. It forms the famous White Cliffs of Dover in Kent, England, as well as their counterparts of the Cap Blanc Nez on the other side of the Dover Strait. The Champagne region of France is mostly underlain by chalk deposits, which contain artificial caves used for wine storage. Some of the highest chalk cliffs in the world occur at Jasmund National Park in Germany and at Møns Klint in Denmark – both once formed a single island.

Formation

Ninety million years ago what is now the chalk downland of Northern Europe was ooze accumulating at the bottom of a great sea. Chalk was one of the earliest rocks made up of sub-microscopic particles to be studied under the electron microscope, when it was found to be composed almost entirely of coccoliths. Their shells were made of calcite extracted from the rich sea-water. As they died, a substantial layer gradually built up over millions of years and, through the weight of overlying sediments, eventually became consolidated into rock. Later earth movements related to the formation of the Alps raised these former sea-floor deposits above sea level.

Composition

The chemical composition of chalk is calcium carbonate, with minor amounts of silt and clay. It is formed in the sea by sub-microscopic plankton, which fall to the sea floor and are then consolidated and compressed during diagenesis into chalk rock.

Uses

Chalk is a source of quicklime by thermal decomposition, or slaked lime following quenching with water.   In southeast England, deneholes are a notable example of ancient chalk pits. Such bell pits may also mark the sites of ancient flint mines, where the prime object was to remove flint nodules for stone tool manufacture. The surface remains at Cissbury are one such example, but perhaps the most famous is the extensive complex at Grimes Graves in Norfolk.

Woodworking joints may be fitted by chalking one of the mating surfaces. A trial fit will leave a chalk mark on the high spots of the corresponding surface. Chalk transferring to cover the complete surface indicates a good fit. Builder's putty also mainly contains chalk as a filler in linseed oil.

Chalk may be used for its properties as a base. In agriculture, chalk is used for raising pH in soils with high acidity. The most common forms are CaCO3 (calcium carbonate) and CaO (calcium oxide). Small doses of chalk can also be used as an antacid. Additionally, the small particles of chalk make it a substance ideal for cleaning and polishing. For example, toothpaste commonly contains small amounts of chalk, which serves as a mild abrasive. Polishing chalk is chalk prepared with a carefully controlled grain size, for very fine polishing of metals.  Chalk can also be used as fingerprint powder.

Previous uses

The traditional uses of chalk have in some cases been replaced by other substances, although the word "chalk" is often still applied to the usual replacements. Blackboard chalk is a substance used for drawing on rough surfaces, as it readily crumbles leaving particles that stick loosely to these surfaces. Although traditionally composed of natural chalk, modern blackboard chalk is generally made from the mineral gypsum (calcium sulfate), often supplied in sticks of compressed powder about 10 cm (4 in) long. Sidewalk chalk is similar to blackboard chalk, but it is shaped into larger sticks and often colored. It is used to draw on sidewalks, streets, and driveways. Tailor's chalk is traditionally a hard chalk used to make temporary markings on cloth, mainly by tailors. However, it is now usually made from talc (magnesium silicate).

Chalk was also traditionally used in recreation. In field sports, such as tennis played on grass, powdered chalk was used to mark the boundary lines of the playing field or court. If a ball hits the line, a cloud of chalk or pigment dust will be visible. In recent years, powdered chalk has been replaced with titanium dioxide.  In gymnastics, rock-climbing, weight-lifting and tug of war, chalk — now usually magnesium carbonate — is applied to the hands and feet to remove perspiration and reduce slipping.

Chalk may also be used as a house construction material instead of brick or wattle and daub: quarried chalk was cut into blocks and used as ashlar, or loose chalk was rammed into blocks and laid in mortar.   There are still houses standing which have been constructed using chalk as the main building material. Most are pre-Victorian though a few are more recent.
Question:
What name, derived from the Greek word for chalk, is given to hydrated calcium sulphate?
Answer:
CaSO4·2H2O

Passage:
Elizabeth Taylor's Top Scandals | TVGuide.com
Elizabeth Taylor's Top Scandals | TVGuide.com
Elizabeth Taylor's Top Scandals
Share on WhatsApp
1 of 8The marrying kind
Taylor loved and loved hard, marrying eight times to seven different men: Conrad Hilton, Michael Wilding, Michael Todd, Eddie Fisher, Richard Burton (twice), John Warner and Larry Fortensky. The marriages spanned 46 years and two of them were born from affairs. Only one didn't end in divorce: her third marriage to film producer Todd, who died in a plane crash. That paved the way for hubby No. 4…
Photo By: UPI/Landov; Jay Scott/Camera Press/Retna; AFP/Getty Images; PA Photos/Landov; Tom Wargacki/WireImage.com; Ron Galella/WireImage.com; Everett Collection
More Photo Galleries
Question:
What is the next in this series: Conrad Hilton, Michael Wilding, Michael Todd, Eddie Fisher, Richard Burton?
Answer:


Prompt:
Please complete the code given below.
switzerland/client/Reporter.py
class Reporter(threading.Thread):
    """ send packet batches and report flow activity """
    flow_activity_interval = 10 # seconds between sending flow updates
    batch_wait_timeout = 1 # maximum time to wait for a full batch

    def __init__(self, parent):
        threading.Thread.__init__(self)
        self.parent = parent
        self.link = parent.link
        self.flow_manager = parent.fm
        self.batch_to_process = parent.fm.batch_to_process
        self.last_flow_activity_update = None
        if self.parent.config.debug_monotonicity:
            self.latest = 0

    def send_flow_stats(self):
        """ send stats about flows """
        pass

    def send_flow_announcements(self):
        """ send information about which flows have been opened/closed """
        if len(self.flow_manager.new_flows) == 0 and \
           len(self.flow_manager.deleted_flows) == 0:
           return # don't bother reporting flow activity if nothing new to report
        new     = [f.summary() for f in self.flow_manager.new_flows]
        deleted = [f.id        for f in self.flow_manager.deleted_flows]

        self.link.send_message("active_flows", [new, deleted])

        for flow in self.flow_manager.new_flows:
            flow.reported = True
        self.flow_manager.new_flows = [ ]
        self.flow_manager.deleted_flows = [ ]

    def send_flow_activity(self):
        self.send_flow_announcements()
        self.send_flow_stats()

    def send_batch(self, batch, flow, now):
        """ send packet batch """
        # have to report flow before sending batches from it
        if not flow.reported:
            # XXX this opening_hash juggling could be cleaned up...
            self.send_flow_announcements()
        hashes = batch.get_hashes()
        timestamp = batch.newest_timestamp
        if self.parent.config.debug_monotonicity:
            if timestamp > self.latest:
                self.latest = timestamp
            else:
                log.debug("send-batch out of order by %gs" % (self.latest - timestamp))
        if flow.inbound:
            self.link.send_message("recd", [flow.id, timestamp, hashes])
        else:
            self.link.send_message("sent", [flow.id, timestamp, hashes])
	#for p in batch.packets:
	#    log.debug("%s %s", p.hash, p.data)
        #log.debug("%d [%d,%d] %d packets for %s", \
        #    now, batch.oldest_timestamp, batch.newest_timestamp, batch.size, flow)

    def run(self):
      try:
        while True:
            self.batch_to_process.acquire()
            # wait for a full batch
            self.batch_to_process.wait(Reporter.batch_wait_timeout)
            # now either there's a full batch or we timed out waiting

            # This looks crazy, but it prevents lots of exceptions during
            # multi-threaded shutdown (the previous line is a long wait)
            try:
                now = int(time.time())
            except:
                return

            # scan batches
            for flow in self.flow_manager.flows.values():
                while flow.queue.has_unsent_batches():
                    batch = flow.queue.get_oldest_unsent_batch()
                    time_batch_open = now - batch.oldest_timestamp
                    # if batch completed or timed out, send it and look at the next
                    if batch.full or time_batch_open > PacketBatch.timeout:
                        self.send_batch(batch, flow, now)
                        flow.queue.mark_oldest_batch_sent()
                        batch.sent = True
                        batch.full = True
                    # otherwise nothing more to do for this flow
                    else:
                        break

            # send flow updates
            if self.last_flow_activity_update == None or \
               now - self.last_flow_activity_update > Reporter.flow_activity_interval:
                self.send_flow_activity()
                self.last_flow_activity_update = now

            self.batch_to_process.release()
      except:
        if traceback:    # only false during interpreter shutdown
          traceback.print_exc()
          self.parent.quit_event.set()
        return
switzerland/client/Cleaner.py
class Cleaner(threading.Thread):
    """Periodically clean up stale packets and expired flows."""

    interval = 10              # seconds between waking up and cleaning

    def __init__(self, parent):
        threading.Thread.__init__(self)
        self.parent = parent

    def run(self):
        try:
            while True:
                time.sleep(Cleaner.interval)
                self.parent.fm.clean(int(time.time()))
        except:
            # Exit-time crash avoidance.
            if self.parent.quit_event.set != None:
                log.info("Cleaner exception:\n" + traceback.format_exc())
                self.parent.quit_event.set()
switzerland/client/TimeManager.py
class TimeManager:

  def __init__(self):

    self.ntp_sysinfo = {}
    self.method = "none"  # what we tell the user about how ntp is working

    # get_time_error is the function Alice will call to get an estimate
    # of the inaccuracy of her clock.  Ideally we use ntpd's measure of
    # root dispersion, but that can be impossible for many reasons, in which
    # case we fall back to something else by switching out this function
    self.get_time_error = self.ntpd_root_dispersion

    log.info("Looking for ntpd...")
    try:
      ntpdc = Popen("ntpdc", stdin=PIPE, stderr=PIPE, stdout=PIPE)
    except:
      log.info("Couldn't run ntpdc.  Will try something else...")
      log.debug(traceback.format_exc())
      self.get_time_error = self.ntpdate_poll
      return

    ntpdc.stdin.write("sysinfo\n")
    ntpdc.stdin.close()
    self.lines = ntpdc.stdout.readlines()
    if not self.lines or "onnection refused" in self.lines[0]:
      log.debug('ntpdc is present but ntpd does not appear to be alive: ("%s").  Will try something else...' % self.lines)
      self.get_time_error = self.ntpdate_poll
      return

    err = ntpdc.stderr.readlines()
    if err:
      log.debug('error running ntpdc: ("%s")' % err)
      self.get_time_error = self.ntpdate_poll
      return

    for line in self.lines:
      pos = line.find(":")
      key = line[:pos].lower()
      val = line[pos+1:].strip()
      self.ntp_sysinfo[key]=val

    mode = self.ntp_sysinfo["system peer mode"]
    if mode == "unspec":
      log.debug("""
      NTP is in "unspec" mode.  This is probably because your system clock
      is wrong by seconds or more.  Switzerland won't work if your clock is
      that wrong.  Unspec mode could also mean that you're offline or that
      you need to give NTP another minute or two to find time servers.  If
      this doesn't work, try stopping NTP, use the ntpdate program to adjust
      the clock and then start NTP again.""")
      log.info("ntpdc is in unspec mode.  Will try something else...")
      self.get_time_error = self.ntpdate_poll
    elif mode != "client":
      log.info("Note that NTP is in mode %s", mode)

  def query_timeserver(self, timeserver):
    '''Query a timeserver using ntpdate and return the clock delta'''
    try:
      regex = re.compile("offset (-?[0-9]+\.[0-9]+) sec")
      executable = "ntpdate"
      # try bin/ntpdate if ntpdate doesn't work
      try:
        Popen(executable, stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=False)
      except:
        executable = os.path.join("bin", "ntpdate")
        try:
          Popen(executable, stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=False)
        except:
          log.error('''Can't even find a copy of "ntpdate" :(''')
          raise UncertainTimeError, "Cannot find an ntpdate executable"

      if platform.system() == "Windows":
        cmd = [executable,"-b","-q",timeserver]
        cmd = " ".join(cmd)
        ntpdate = Popen(cmd, stdin=PIPE, stderr=PIPE, stdout=PIPE, shell=True)
      else:
        cmd = [executable,"-q",timeserver]
        ntpdate = Popen(cmd, stdin=PIPE, stderr=PIPE, stdout=PIPE)
      ntpdate_lines = ntpdate.stdout.read()
      match = regex.search(ntpdate_lines)
      if not match:
        log.error("Failed to parse the output from ntpdate.")
        log.debug(ntpdate_lines)
        return None
      delta = float(match.group(1))
      return abs(delta)

    except UncertainTimeError:
      # let this pass upwards
      raise
    except:
      log.warn("Weird but possibly non-fatal error:\n"+traceback.format_exc())
      return None

  def ntpdate_poll(self):
    '''An alternative method for finding the root dispersion'''
    servers = ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org"]
    try:
      results = [self.query_timeserver(s) for s in servers]
      results = filter(lambda r : r != None, results)
      if results:
        log.debug("ntpdate poll results are: %s" % results)
        log.info("Believing the maximum clock error reported by %d timeservers"\
                  % len(results))
        log.debug("Not quite as good as a working ntpd, but this should be ok")
        self.method = "ntpdate -q"
        return max(results)
      else:
        log.info("We haven't succeeded in polling any NTP servers, either.")
        raise UncertainTimeError
    except UncertainTimeError:
      # let this pass upwards
      raise
    except:
      log.debug("ntpdate_poll error:\n%s" % traceback.format_exc())
      raise UncertainTimeError

  def ntpd_root_dispersion(self):
    try:

      mode = self.ntp_sysinfo["system peer mode"]
      self.method = "ntpd, root dispersion measured with ntpdc (%s mode)" % mode
      return float(self.ntp_sysinfo["root dispersion"].split()[0])
    except KeyError:
      # try to use ntpdate to calculate the root dispersion by finding the max
      # deviation as reported by 3 time servers
      log.warning("Couldn't find root dispersion in ntpdc output.  Trying ntpdate...")
      return self.ntpdate_poll()
switzerland/client/AliceLink.py
class AliceLink(Protocol.Protocol):
  "Extend the protocol base class for Alice's end."

  def __init__(self, quit_event, parent, config):
    self.in_messages = Messages.alice_in_messages
    self.out_messages = Messages.alice_out_messages
    try:
      self.flow_manager = parent.fm
    except AttributeError:
      self.flow_manager = None
    self.parent = parent
    self.config = config
    self.quit_event = quit_event
    self.dangling_note = False
    self.forged_notification = True
    try:
      self.server_ip = s.gethostbyname(config.host)
    except:
      log.error("DNS error looking up %s" % config.host)
      sys.exit(1)

    self.socket = s.socket(s.AF_INET, s.SOCK_STREAM)
    log.info("Connecting to %s:%s", config.host, `config.port`)
    self.socket.settimeout(10.0)
    try:
      self.socket.connect((self.server_ip, config.port))
    except:
      log.error("couldn't connect to %s:%s", config.host, `config.port`)
      sys.exit(1)
    self.socket.settimeout(40)

    self.peer = (config.host, config.port)
    self.time_connected = time.time()
    self.member_queue = []  # members to be added once we public_ip is known

    Protocol.Protocol.__init__(self, log, config.seriousness, private_ip=config.private_ip)
    # up until now it has been safe to send messages on the socket because
    # we are still blocking the caller thread

  def run(self):
    excepted = False
    try:
      Protocol.Protocol.run(self)
    except:
      print traceback.format_exc()
      excepted=True
    if not excepted:
      log.info("Adieu...")
    self.quit_event.set()
    self.ready.set() # to hurry Alice up

  def handshake(self):
    """Confirm that we are talking to a Switzerland server"""

    self.socket.send(Protocol.handshake1)

    self.socket.settimeout(30)
    try:
      msg = self.socket.recv(len(Protocol.handshake2))
    except s.timeout:
      self.debug_note("Timeout during handshake", seriousness=11)
      return False
    except s.error, e:
      log.error("error during handshake: %s" % e)
      return False
    except:
      log.error("error during handshake")
      return False

    if msg[:-2] == Protocol.no_common_version[:-2]:
      log.error("Server refuses to speak protocols before version %d; we need version %d" % (Protocol.parse_version(msg[-2:]), Protocol.protocol_version))
      return False

    elif msg[:-2] != Protocol.handshake2[:-2]:
      log.error("Handshake failed, wasn't expecting:\n" + msg)
      return False

    log.debug("Started session with version %d" % Protocol.parse_version(msg[-2:]))
    self.socket.setblocking(1)
    return True

  def setup(self):
    "Handshake successful; now do initial housekeeping"
    self.send_myip()

  def lookup_flow_by_id(self, flow_id, caller):
    """
    Called in handle_forged_{in,out}
    Return the flow, or None if we fail.
    "caller" is modified-{in,out} & is only used for error reporting.
    """
    if flow_id not in self.flow_manager.flow_id_to_address:
      msg = "stale %s flow id %s" % (caller, `flow_id`)
      log.info(msg)
      self.send_message("error-cont", [msg])
      return None

    flow_addr = self.flow_manager.flow_id_to_address[flow_id]
    if flow_addr not in self.flow_manager.flows:
      msg= "bad %s flow addr %s in id to addr dict" % (caller, `flow_addr`)
      log.error(msg)
      self.send_message("error-cont", [msg])
      return None

    flow = self.flow_manager.flows[flow_addr]
    return flow

  def handle_forged_out(self, args, seq_no):
    """ Send context surrounding a forged hash """
    if self.flow_manager == None:
      # we're in a test case
      return
    self.flow_manager.lock.acquire()
    try:
      flow_id, forgeries = args
      log.warn("Switzerland has informed us of %d modified/spoofed outbound packets in flow #%d" % (len(forgeries), flow_id))
      self.forged_note()
      flow = self.lookup_flow_by_id(flow_id, "modified-out")
      self.nat_firewall_warnings(flow)
      contexts = {}
      reports = {}
      for wanted in forgeries:
        hash = wanted[0]             # hash of forged packet
        receipt_context = wanted[1]
        filename = self.parent.pcap_logger.log_forged_in(receipt_context, flow_id)
        ctxt, report = flow.get_fo_context_and_diff(receipt_context, self.parent)
        if report:
          log.error("Here is what we know about these modified packets:\n%s\n" %
                    fmt_forged_report(report))
          reports[hash] = report
        else:
          log.error("This packet appears to have been injected")
          log.error("(at least, we couldn't find a packet we're sure was modified)")
          log.error("(AND THIS CODE SHOULDN'T RUN)")
        if ctxt:
          contexts[hash] = ctxt
          self.parent.pcap_logger.log_forged_out(ctxt, filename)
        else:
          log.error("No modified-out context for %s" % hexlify(hash))
          contexts[hash] = None
      self.send_message("fo-context", [contexts, reports], reply_seq_no=seq_no)

    finally:
      self.flow_manager.lock.release()

  def handle_forged_in(self, args, seq_no):
    """ Send context about forged packets whose batches have exactly
        the newest_timestamps specified in timestamps. """
    self.flow_manager.lock.acquire()
    try:
      try:
        flow_id, packets_wanted = args
        log.warn("Switzerland has informed us of %d modified inbound packets in flow #%d" % (len(args[1]), flow_id))
        self.forged_note()
        flow = self.lookup_flow_by_id(flow_id, "modified-in")
        if not flow or self.nat_firewall_warnings(flow) == -1:
          # errors have already been reported
          return

        contexts = {}
        out_filenames = []
        for ts,hash in packets_wanted:
          context = contexts[hash] = flow.get_fi_context(ts, hash)
          if not context:
            log.warn("No context was available for hash %s" % hexlify(hash))
            out_filenames.append("")
          else:
            fn = self.parent.pcap_logger.log_forged_in(context, flow_id)
            out_filenames.append(fn)

        self.send_message("fi-context", [contexts], reply_seq_no=seq_no,\
                          data_for_reply=out_filenames)
      except:
        log.error("exception in handle_forged_in:")
        log.error(traceback.format_exc())
    finally:
      self.flow_manager.lock.release()

  def forged_note(self):
    """
    Point the user towards some (hopefully) helpful documentation about the
    forged -in and -out PCAPs.  But only do this once!
    """
    if not self.forged_notification:
      log.warn("For information about Switzerland's modified/forged PCAP "+\
               "records, please see")
      log.warn("http://switzerland.wiki.sourceforge.net/forged")
      self.forged_notification = True

  def nat_firewall_warnings(self, flow):
    """
    Warn the user if they are seeing reports of modified packets when there
    are NAT firewalls along the path, since those NATs are mischievous.
    Return -1 if we encounter & report a fatal error along the way.
    """

    if flow.inbound:
      them = flow.src_ip
    else:
      them = flow.dest_ip
    them_p = s.inet_ntoa(them)
    try:
      peer_firewalled = self.flow_manager.is_peer_firewalled(them)
    except:
      msg = "Can no longer find " + them_p + " in peer records ("
      msg += `self.flow_manager.peers` + ")"
      log.error(msg)
      self.send_message("error-cont", [msg])
      return -1
    if not (peer_firewalled or self.firewalled):
      return
    msg = "(Note that "
    if self.firewalled:
      msg += "you are behind a NAT firewall"
      if peer_firewalled:
        msg += " and "
    if peer_firewalled:
      msg += "this other machine (at %s) is behind a NAT firewall" % them_p
    msg += ", so it is likely that these modifications are being made "
    msg += "by firewalls, not ISPs)"
    log.warn(msg)

  def handle_forged_details(self, args, reply_seq_no):
    """
    A "forged-details" message is the followup to for forged-in; it shows
    us Alice's side of the story.
    """
    meta = args[0]
    in_reply_to, remembered = meta
    id = args[1]
    msgs = args[2]

    if in_reply_to != "fi-context":
      self.protocol_error("reply %d should not be a forged-details message\n" % reply_seq_no)
      sys.exit(0)

    out_filenames = remembered

    for filename, (timestamp, context, report) in zip(out_filenames, msgs):
      if filename:
        # This nonsense ensures we're prettier for the users
        log.error("Report on forged/modified packet:\n%s\n" % \
                  fmt_forged_report(report))
        if context:
          self.parent.pcap_logger.log_forged_out(context, filename)
        else:
          log.error("no outbound context to write to " + filename)

  def dummy_handle_forged_in(self, args, seq_no):
    "This variant of handle_forged_in is only used for testing"
    self.debug_note("Heard about %d dummy forged packets: %s" % \
                    (len(args[1]), `args[0]`), 2)
    forgeries = args[1]
    contexts = {}
    for timestamp,hash in forgeries:
      contexts[hash] = [(1,"packet1"), (2,"packet2"),(3,"x" + `timestamp`)]
    self.send_message("fi-context", [contexts], reply_seq_no=seq_no)

  def handle_new_members(self, args):
    new_peers = args[0]
    try:
      self.flow_manager.listen_for(new_peers)
    except:
      # it's most likely this is due to an invalid new_peers list
      self.protocol_error("Problem with new-members: %s\n" % new_peers)
      raise

  def handle_farewell(self, args):
    old_peer = args[0]
    try:
      if self.flow_manager:
        self.flow_manager.farewell(old_peer)
    except:
      self.protocol_error("Problem with farwell: %s\n" % old_peer)
      raise

  def send_myip(self):
    "Figure out the local IP address, and send it."
    if self.config.force_public_ip: # if force_public_ip is set, assume this public ip
      self.send_message("my-ip", [[self.private_ip, self.config.force_public_ip]])
    else:
      self.send_message("my-ip", [[self.private_ip]])

  def handle_public_ip(self, args):
    "Switzerland tells alice her publicly visible ip"
    stored_data = args[0]
    log.debug("Switzerland says our public ip is %s", `args[1]`)
    self.public_ip = args[1]
    self.firewalled = self.public_ip != self.private_ip
    log.info("Private IP: %s ; Public IP: %s", `self.private_ip`, `self.public_ip`)
    self.now_ready()

  def handle_dangling_flow(self, args):
    "Switzerland has failed to matchmake our view of this flow with out peers"
    msg = "Switzerland is unable to test flow #%d.  " % args[0]
    if not self.dangling_note:
      msg += "Reasons for this may include:\n"
      msg += " - Modifications to the first packet in the flow (hash 0x%s)\n" % \
             hexlify(args[1])
      # XXX we could make the inclusion of some of these lines conditional on
      # whether the peer is actually firewalled
      msg += " - The flow is not actually with the other Switzerland client\n"
      msg += "   (it could be with their firewall, another machine on their LAN,\n"
      msg += "   or an impostor)\n"
      msg+=" - Alice and Bob seeing a different packet as the first packet in the\n"
      msg+="   flow (most likely if the flow was active before switzerland started)\n"
    else:
      msg += "(opening hash 0x%s)" % hexlify(args[1])
    log.warn(msg)
    self.dangling_note = True
    # XXX we should also take steps to prevent the flow manager from send sent
    # and recd messages for this flow

  def debug_ip_id(self, args):
    ip_id = args[0]
    print "\n\n\n\n"
    print "Debugging ip_id", hexlify(ip_id)
    print "ip_id table:", self.flow_manager.ip_ids
    #if ip_id in self.flow_manager.scapy_ip_ids:
    #  print "ip_id is in the scapy table:",self.flow_manager.scapy_ip_ids[ip_id]

    #else:
    #  print "ip_id is not in scapy table"
    try:
      print "Found:", hexlify(self.flow_manager.ip_ids[ip_id])
    except:
      print "Unable to find ip_id in flow_manager.ip_ids!!!"
    print "\n\n\n\n"

  def now_ready(self):
    if self.member_queue:
      map (self.handle_new_members, self.member_queue)
    Protocol.Protocol.now_ready(self)

  def determine_response(self, msg_type, args, seq_no, reply_seq_no):

    # First look for messages common to Alice & Switzerland
    if Protocol.Protocol.determine_response(self, msg_type, args, seq_no, reply_seq_no):
      return True

    # Okay, that didn't work...
    if msg_type == "forged-in":
      self.handle_forged_in(args, seq_no)
      return True
    elif msg_type == "new-members":
      if self.flow_manager:  # only false in unit tests
        if self.ready.isSet():
          self.handle_new_members(args)
        else:
          self.member_queue.append(args)
      return True
    elif msg_type == "forged-out":
      self.handle_forged_out(args, seq_no)
      return True
    elif msg_type == "forged-details":
      self.handle_forged_details(args, reply_seq_no)
      return True
    elif msg_type == "farewell":
      self.handle_farewell(args)
      return True
    elif msg_type == "debug-ipid":
      self.debug_ip_id(args)
      return True
    elif msg_type == "public-ip":
      self.handle_public_ip(args)
      return True
    elif msg_type == "flow-status":
      log.info(args[0])
      return True
    elif msg_type == "dangling-flow":
      self.handle_dangling_flow(args)
      return True
    else:
      return False

    return True   # Tricksy

  def tst(self):
    self.send_message("ping")
    self.send_message("sent", ["flow","time",[1,2,4]])
    time.sleep(1)
    self.send_message("sent", ["flow","time",[1,2,5]])
    time.sleep(2)
    self.send_message("sent", ["flow","time",[1,2,6]])
    self.send_message("ping")
    self.send_message("ping")
    self.send_message("ping")
    self.send_message("signoff")
switzerland/client/TimeManager.py
class UncertainTimeError(Exception):
  pass
switzerland/common/PcapLogger.py
class PcapLogger:
  """
  This device organises the elaborate switzerland logging structure, which
  is a standard log containing references to a directory of incident-specific
  pcap log files.
  """

  # XXXXXX fix this default before release
  def __init__(self, log_dir):
    self.lock = threading.RLock()
    self.make_or_check_directory(log_dir, "log dir")
    #ts = self.timestamp()
    self.log_dir = log_dir
    # XXX do not expect this to be secure if log_dir is world writeable
    # on a multi-user system
    self.make_or_check_directory(self.log_dir, "pcap log dir")

  def make_or_check_directory(self, path, alias="directory"):
    alias += " "
    try:
      os.mkdir(path, 0755)
    except OSError, e:
      assert not os.path.islink(path), alias + path + " should not be a symlink!"
      assert os.path.isdir(path), "cannot create " + alias + path + ":\n"+`e`

  def new_flow(self, flow, id):
    log.info("New bidirectional flow %d : %s\n" % (id,`flow`))

  def timestamp(self, t=None):
    if t == None:
      t = time.time()
    frac = `t - int(t)`[2:5]
    return time.strftime("%Y%m%d%H%M%S", time.localtime(t)) + frac

  def log_forged_in(self, context, id):
    self.lock.acquire()
    try:
      in_path, out_path = self.__gen_filenames(context)
      pcap = PcapWriter(in_path)
      self.log_to_pcap(context, pcap)
      pcap.file.close()
    finally:
      self.lock.release()
    return out_path

  def log_forged_out(self, context, out_path):
    self.lock.acquire()
    try:
      pcap = PcapWriter(out_path)
      self.log_to_pcap(context, pcap)
      pcap.file.close()
    finally:
      self.lock.release()

  def __gen_filenames(self, context):
    "Determine the filenames for the inbound and outbound logs"

    ts = self.timestamp(context[0][0])
    filename1 = ts + "-in.pcap"
    filename2 = ts + "-out.pcap"
    path1 = self.log_dir + os.path.sep + filename1
    path2 = self.log_dir + os.path.sep + filename2
    n = 0
    while os.path.exists(path1):
      # This inbound filename has already been used; find another
      filename1 = ts + "-"  + `n` + "-in.pcap"
      filename2 = ts + "-"  + `n` + "-out.pcap"
      path1 = self.log_dir + os.path.sep + filename1
      path2 = self.log_dir + os.path.sep + filename2
      n +=1
    log.info("Recording inbound modified packets & context in %s\n" % path1)

    return (path1,path2)

  def log_to_pcap(self, packets, pcap_log):
    ## consider adding linktype= to this:

    for timestamp, hash, packet in packets:
      # be careful what we feed to scapy!
      if type(packet) != types.StringType:
        raise LoggerError, "Packet data %s is not a string!\n" % packet

      try:
        pcap_log.write(packet,timestamp)
      except:
        log.error("Tripped on %s", `pcap_log`)
        raise
      pcap_log.file.flush()
switzerland/client/FlowManager.py
class FlowManager:
    "FlowManager tracks active Flows (and also the Peers who are in our circle"

    def __init__(self, config, parent=None):
        self.flows = { }
        self.flow_id_to_address = { }
        # jered used these to implement the Reporter:
        self.new_flows = [ ]
        self.deleted_flows = [ ]
        # I'mma use these to implement the API:
        self.api_new_flows = []

        self.unique_id = 0
        self.config = config
        self.private_ip = s.inet_aton(config.private_ip)
        self.lock = threading.RLock()
        self.batch_to_process = threading.Condition(self.lock)
        self.peers = {}
        self.parent = parent

        self.ip_ids = {}        # for debugging

    def farewell(self, old_peer):
        "A peer has left. Remove any outstanding flows with it."
        self.lock.acquire()
        try:
            ip = s.inet_aton(old_peer)
            try:
                del self.peers[ip]
            except KeyError:
                msg = "Farewell message for non-existent peer %s" % old_peer
                log.error(msg)
                self.parent.link.send_message("error-cont", [msg])

            for f in self.flows.values():
                if f.src_ip == ip or f.dest_ip == ip:
                    f.marked_for_deletion = True
            log.info("Peer %s has disconnected" % old_peer)
        finally:
            self.lock.release()

    def listen_for(self, new_peers):
        "Start sniffing packets from these peers."
        self.lock.acquire()
        try:
            note = "Listening for traffic with peers: "
            for p,firewalled in new_peers:
                # XXX add a test for private IPs here
                # s.inet_ntoa(p) asserts that p is a valid peer address
                new = s.inet_ntoa(p)
                assert s.inet_ntoa(p) != self.parent.link.private_ip
                assert s.inet_ntoa(p) != self.parent.link.public_ip

                self.peers[p] = Peer(firewalled)
                note += new
                if firewalled: note += '*'
                note += ", "
            note = note[:-2] # chop off the last comma & space
            log.info(note)
        finally:
            self.lock.release()

    def is_peer_firewalled(self, peer):
        "Here to let a packet know if its peer is firewalled"
        self.lock.acquire()
        try:
          return self.peers[peer].firewalled
        finally:
          self.lock.release()

    def is_local_flow(self, packet):
        """
        determine if this flow is to or from the local machine (Alice)
        so we can filter out broadcast traffic.
        """
        if self.config.ignore_nonlocal_packets:
          if packet.dest_ip == self.private_ip and packet.source_ip == self.private_ip:
            log.warning("sniffing packets from self!")
          if packet.source_ip == self.private_ip or packet.dest_ip == self.private_ip:
            return True
          else:
            return False
        else:
          raise NotImplementedError, "Promiscuity not implemented"

        return True

    def mark_if_firewalled(self, packet):
        """
        flag whether peer in this packet is firewalled
        """
        if packet.dest_ip == self.private_ip:
          peer_ip = packet.source_ip
        elif packet.source_ip == self.private_ip:
          peer_ip = packet.dest_ip
        else:
          raise NotImplementedError, "Promiscuity not implemented"

        try:
          packet.peer_firewalled = self.is_peer_firewalled(peer_ip)
        except KeyError:
          # we don't know if they're firewalled, since they're
          # not a true peer
          packet.peer_firewalled = True

    def is_in_circle(self, packet):
        """
        determine if a packet is to or from a host in Alice's circle.
        packets to hosts outside Alice's circle will not be logged (unless
        config.filter_packets is False), however statistics will be kept
        and sent to switzerland (anonymously)
        """
        if not self.config.filter_packets:
          return True

        self.lock.acquire()
        try:
            if (packet.dest_ip not in self.peers) and \
               (packet.source_ip not in self.peers):
                return False
        finally:
            self.lock.release()

        return True

    def archive_packet(self, packet):
      global a_lock
      global archives
      if not self.config.keep_archives:
        return
      a_lock.acquire()
      try:
          thread = self.parent.link.thread_no()
          archive = archives.setdefault(thread,{})
          hash = packet.get_hash()
          archive.setdefault(hash,[]).append(packet)

      finally:
          a_lock.release()

    def create_flow(self, packet, flow_addr):
        """ create a new flow with given initial packet and flow id string """
        opening_hash = packet.get_hash()
        flow = AliceFlow(packet.dest_ip == self.private_ip, # inbound flow?
            packet.source_ip, packet.source_port,
            packet.dest_ip, packet.dest_port, packet.proto,
            packet.timestamp, opening_hash, self.is_in_circle(packet))
        flow.id = self.unique_id # internal serial# for flow, != flow_addr
        self.unique_id += 1
        if flow.in_circle: # only report flows in circle to switzerland
            self.new_flows.append(flow)
            self.api_new_flows.append(flow)
            log.info("Attempting to test flow #%d %s" % (flow.id, flow.__str__()))
            log.debug("(flow opens with hash: %s\npacket data: %s" % \
                (hexlify(opening_hash), hexlify(packet.data.tostring())))
            log.debug("masked from: %s)" % hexlify(packet.original_data))
        self.flows[flow_addr] = flow
        # map IP datagram flow info to our internal flow id
        self.flow_id_to_address[flow.id] = flow_addr

    def handle_packet(self, packet):
        """
        called for each incoming packet. if packet destined for alice from
        host in circle, find existing flow or detect new flow, then queue
        for that flow.  return True if the packet completes a PacketBatch,
        False otherwise.
        """
        self.lock.acquire()
        try:
            assert isinstance(packet, Packet.Packet), 'expecting Packet'
            if not self.is_local_flow(packet): # ignore packets not from/to alice
                return False
            self.mark_if_firewalled(packet)
            self.archive_packet(packet)

            flow_addr = packet.flow_addr()
            if not self.flows.has_key(flow_addr): # create flow if it doesn't already exist
                self.create_flow(packet, flow_addr)

            flow = self.flows[flow_addr]
            assert flow != None, 'expecting to find flow'

            # track flow data
            flow.activity = True
            flow.time_last_active = packet.timestamp
            flow.bytes_transferred += len(packet)
            flow.packets_transferred += 1

            if flow.in_circle:
                return flow.queue.append(packet)
            return False
        finally:
            self.lock.release()

    def clean(self, now):
        """ remove inactive flows or stale packets in active flows """
        self.lock.acquire()
        try:
          for f in self.flows.keys():
              flow = self.flows[f]
              inactive = now - flow.time_last_active
              if inactive > Flow.timeout or flow.marked_for_deletion:
                  log.debug("Flow %s timed out, deleting", `flow`)
                  flow.active = False
                  if flow.in_circle: # switzerland only knows about this flow if it's in alice's circle
                      self.deleted_flows.append(flow)
                  del self.flow_id_to_address[flow.id]
                  del self.flows[f]
              else:
                  flow.clean(now)
        finally:
          self.lock.release()
switzerland/client/AliceConfig.py
class AliceConfig:
    def __init__(self, getopt=False, **keywords):

        # all of the variables above become attributes of AliceConfig;
        # we pick the value passed to this function (if there was one)
        # or the default value otherwise

        self.private_ip = None

        for variable in alice_options.keys():
            if variable in keywords:
                self.__dict__[variable] = keywords[variable]
            else:
                self.__dict__[variable] = alice_options[variable][0]

        if getopt:
            self.get_options()

        # Defaults remaining if getopt hasn't set things:
        if self.interface == None and getopt:
            self.interface = local_ip.get_interface()
            log.info("interface is now %s", self.interface)

        if self.force_private_ip and self.private_ip == None: # could have been set by getopt?
            self.private_ip = force_private_ip
        if self.private_ip == None:
            self.private_ip = local_ip.get_local_ip(self.interface)

    def get_options(self):
        if len(sys.argv) > 1 and sys.argv[1] == "help":
            self.usage()
        try:
            (opts, args) = \
                getopt.gnu_getopt(sys.argv[1:], 's:p:i:l:u:L:P:f:b:hqv',
                    ['server=', 'port=', 'interface=', 'ip=', 'help',
                    'private-ip=', 'public-ip=', 'logfile=', 'pcap-logs=',
                    'quiet', 'uncertain-time', 'verbose', 'buffer='])
        except getopt.GetoptError:
            self.usage()

        for opt, arg in opts:
            if opt in ('-s', '--server'):
                try:
                    if ":" in arg:
                        self.host, self.port = arg.split(":")
                        self.port = int(self.port)
                    else:
                        self.host = arg
                except ValueError:
                    self.usage(False)
                    print "Invalid argument for the", opt, "flag (specify a",
                    print "server to connect to)"
                    sys.exit(1)
            elif opt in ('-f', '--public-ip'):
                self.force_public_ip = arg
            elif opt in ('-p', '--port'):
                try:
                    self.port = int(arg)
                except ValueError:
                    self.usage(False)
                    print "Invalid argument for the", opt, "flag (need a",
                    print "port number"
                    sys.exit(1)
            elif opt in ('-i', '--interface'):
                self.interface = arg
            elif opt in ('-b', '--buffer'):
                try:
                    self.packet_buffer_size = int(arg)
                    assert(self.packet_buffer_size > 1)
                except:
                    print "Invalid packet buffer size (suggestion: 25000 or more)"
                    sys.exit(1)
            elif opt in ('-l', '--ip', '--private-ip'):
                self.private_ip = arg
            elif opt in ('-L', '--logfile'):
                self.logfile = arg
            elif opt in ('-P', '--pcap-logs'):
                self.pcap_logdir = arg
            elif opt in ('-h', '--help'):
                self.usage()
            elif opt in ('-q', '--quiet'):
                self.quiet = True
            elif opt in ('-u', '--uncertain-time'):
                self.allow_uncertain_time = True
                try:
                    self.manual_clock_error = float(arg)
                except ValueError:
                    self.usage(False)
                    print "Invalid argument for the", opt, "flag (please",
                    print "specify the maximum error of your clock in seconds)"
                    sys.exit(1)
            elif opt in ('-v', '--verbose'):
                self.log_level -= (logging.INFO - logging.DEBUG)

    def usage(self, exit=True):
        print
        print "%s [OPTIONS]" % sys.argv[0]
        print
        print "This is the client for EFF's Switzerland network traffic auditing system"
        print
        print "  -h, --help                 Print usage info"
        print "  -s, --server <host[:port]> Switzerland server [and port]"
        print "  -p, --port <port number>   Switzerland server port"
        print "  -i, --interface <iface>    Interface on which to monitor traffic"
        print "  -l, --private-ip <ip>      (LAN) ip address of monitored interface"
        print "  -f, --public-ip <ip>       Force a public IP address.  Useful if you want to"
        print "                             tunnel to a server using ssh, tor,  a proxy, etc. "
        print "                             NB:this is disallowed by the default server config"
        print "  -u, --uncertain-time       Work without NTP.  This is dangerous;"
        print "       <time in seconds>     acurately specify the error in your system clock"
        print '  -L, --logfile <file>       Write a copy of the output to <file>. "-" for none'
        print "                             (defaults to " + default_logfile + ")"
        print "  -P, --pcap-logs <dir>      Sets the directory to which PCAPs of modified"
        print '                             packets will be written. "-" for none.'
        print "                             (defaults to " + default_pcap_logdir + ")"
        print "  -q, --quiet                Do not print output"
        if sys.argv[0].endswith('WebGUI.py'):
            print "  -a, --webaddr              Sets the ip address to run the Switzerland client"
            print "                             web server (defaults to 0.0.0.0)"
            print "  -w, --webport              Sets the port to run the Switzerland client"
            print "                             web server (defaults to 8080)"
            print "  -F, --fake                 Runs WebGUI in an off-network demonstration mode"
            print "                             with fake data"
        print
        if exit: sys.exit(0)

    def check(self):
        if not self.host:
            self.host = "switzerland.eff.org"
            log.info("no switzerland server specified, defaulting to %s" % self.host)

        # check for a valid ip address
        if self.private_ip == None:
            log.error("Switzerland wasn't able to determine your local IP address.")
            log.error("Make sure you're online; if you are, use the -l flag to specify you IP")
            sys.exit(1)
        try:
            s.inet_aton(self.private_ip)
        except s.error:
            log.error("invalid local address format %s", `self.private_ip`)
            sys.exit(1)

        if self.pcap_logdir and self.pcap_logdir != "-":
            try:
                st = os.stat(self.pcap_logdir)
                if not stat.S_ISDIR(st[stat.ST_MODE]): # not a directory
                    log.error("%s isn't a directory", self.pcap_logdir)
                    log.error('Please make this a directory, or use "-P -" for no logging')
                    sys.exit(1)
            except OSError, e:
                if e.errno == errno.ENOENT: # no such file or directory
                    log.warn("PCAP log directory %s doesn't exist", self.pcap_logdir)
                    log.warn("trying to create it")
                    dir = os.path.split(self.pcap_logdir)
                    create = [dir[1]]
                    while ((not os.path.exists(dir[0])) and (not dir[0] == os.path.dirname(dir[0]))):
                        dir = os.path.split(dir[0])
                        create.insert(0, dir[1])
                    try:
                        dir=dir[0]
                        for newdir in create:
                            dir=os.path.join(dir, newdir)
                            os.mkdir(dir)
                    except OSError:
                        log.error("can't create PCAP log directory")
                        log.error('Please create it, or use "-P -" for no logging')
                        sys.exit(1)
                else: # can't stat, that's probably bad
                    log.error("can't stat PCAP log directory %s: %s", self.pcap_logdir, str(e))
                    log.error('Use "-P -" for no logging')
                    sys.exit(1)
            if not util.writable(self.pcap_logdir):
                log.error("Cannot write to PCAP log directory %s", self.pcap_logdir)
                log.error("Change its permissions or specify another directory with -P")
                log.error('Use "-P -" for no logging')
                sys.exit(1)
switzerland/client/Tracerouter.py
class Tracerouter(threading.Thread):
    def __init__(self):
        self.ready = False
        self.tr_cmd = CALCULATING
        self.routes = {}
        self.q_work = threading.Condition()
        self.queue = []
        threading.Thread.__init__(self)
        self.setDaemon(True)

    def run(self):
        self.setup()
        self.do_traceroutes()

    def setup(self):
        if platform.system() == "Windows":
            tr_cmd = ["tracert"]
            use_shell = True
        else:
            tr_cmd = ["traceroute"]
            use_shell = False

        try:
            run = Popen(["mtr", "--report", "-n", "localhost"],
                        shell=use_shell, stdout=PIPE)
            test = run.stdout.read()
            if "1" in test:
                tr_cmd = ["mtr", "--report", "--report-cycles", "2"]
                # print "mtr works..."
        except:
            try:
                run = Popen(tr_cmd + [ "localhost"],
                            shell=use_shell, stdout=PIPE)
                test = run.stdout.read()
            except:
                print traceback.format_exc()
                tr_cmd = None
        self.tr_cmd = tr_cmd
        self.use_shell = use_shell
        self.ready = True
        print "ready"

    def do_traceroutes(self):
        while True:

            try:
                target_ip = self.get_from_queue()
                run = Popen(self.tr_cmd + ["-n", target_ip], stdout=PIPE)
                self.routes[target_ip] = run.stdout.read()
            except:
                traceback.print_exc()

    def get_from_queue(self):
        # Wait til there's something in the queue
        self.q_work.acquire()
        if len(self.queue) == 0:
            self.q_work.wait()
        else:
            self.q_work.release()

        # Grab the next traceroute target off the queue
        target_ip = self.queue[0]
        del self.queue[0]
        print "got", target_ip
        return target_ip

    def enqueue(self, target):
        self.q_work.acquire()
        try:
            self.queue.append(target)
            self.routes[target] = "Waiting"
            self.q_work.notify()
        finally:
            self.q_work.release()

    def route_to(self, target_ip):
        if not self.tr_cmd:
            return False
        elif self.tr_cmd == CALCULATING:
            self.enqueue(target_ip)
            return "Waiting"
        elif target_ip in self.routes:
            return self.routes[target_ip]
        else:
            self.enqueue(target_ip)
            # This *probably* means we return "waiting", unless things happen
            # very quickly
            return self.routes[target_ip]
switzerland/client/PacketListener.py
class PacketListener(threading.Thread):
    """ sniff incoming IP datagrams on a network interface or from a save file.
        pass packets off to FlowManager. """

    def __init__(self, parent):
        """self.parent.fm: FlowManager to receive packets"""
        threading.Thread.__init__(self)
        self.parent = parent
        self.packets_in_buf = parent.config.packet_buffer_size
        self.entry_size = packet_size + packlen_size + ts_size + valid_size + \
                          padding2_size
        self.buffer_size = self.packets_in_buf * self.entry_size
        self._zcat_pipe = None
        self._zcat = None
        self.tmpfile = None
        self.lock = threading.RLock()
        self.latest = 0
        self.max_timewarp = 0
        self._skew = parent.config.skew
        self.kernel_tweaked = False
        self.frag_warn = False # have we warned about fragmented packets?
        self.done = threading.Event() # are we done replaying an
                                      # offline capture?
        self.half_done = False  # True when FastCollector has finished
                                # healthily but we haven't read all the
                                # packets from the buffer yet

        if platform.system() == 'Windows':
          self.sniffer_thread = os.path.sep.join(['FastCollector.exe'])
          self.sniffer_thread2 = os.path.sep.join(['bin','FastCollector.exe'])
        else:
          self.sniffer_thread = os.path.sep.join(['FastCollector'])
          self.sniffer_thread2 = os.path.sep.join(['bin','FastCollector'])
        if parent.config.pcap_playback:
          self.live = False
          self.target = self.filename_magic(parent.config.pcap_playback)
        else:
          assert parent.config.interface, "Can't create PacketListner with neither a pcap file nor an interface"
          self.live = True
          self.target = parent.config.interface

    def minimize_packet_loss(self):
        """
        Platform-specific code to maximise buffer/queue space in the kernel
        captured packets, to minimize the risk that the kernel drops packets
        before they get to the sniffer.
        """
        p = platform.system()
        # Implementing the recommendations from
        # http://www.net.t-labs.tu-berlin.de/research/hppc/
        if p[-3:] == "BSD" or p == "Darwin":
          cmd = ["sysctl","-w","net.bpf.bufsize=10485760"]
          try:     # Recent FreeBSDs
            proc = Popen(cmd, stdin=PIPE, stdout=PIPE)
            assert proc.wait() == 0
            cmd[2] = "net.bpf.maxbufsize=10485760"
            proc = Popen(cmd, stdin=PIPE, stdout=PIPE)
            assert proc.wait() == 0
          except:  # Older FreeBSDs
            cmd[2] = "debug.bpf_bufsize=10485760"
            proc = Popen(cmd, stdin=PIPE, stdout=PIPE)
            assert proc.wait() == 0
            cmd[2] = "debug.maxbpf_bufsize=10485760"
            proc = Popen(cmd, stdin=PIPE, stdout=PIPE)
            assert proc.wait() == 0

        elif p == "Linux":
          vars = [("/proc/sys/net/core/rmem_default", "33554432"),
                  ("/proc/sys/net/core/rmem_max", "33554432"),
                  ("/proc/sys/net/core/netdev_max_backlog", "10000")]
          self.old_kernel_state = []
          for file, value in vars:
            # save kernel state before overwriting it
            f = open(file, 'r')
            self.old_kernel_state.append((file, f.read()))
            f.close()
            f = open(file, 'w')
            f.write(value)
            f.close()
          self.kernel_tweaked = True

        # on Windows, we should call pcap_setbuff from inside FastCollector.

    def restore_kernel_state(self):
        p = platform.system()
        if p == "Linux":
            for file, value in self.old_kernel_state:
                # save kernel state before overwriting it
                f = open(file, 'w')
                f.write(value)
                f.close()
            self.kernel_tweaked = False

    def launch_collector(self):
        """
        Launch the PacketCollector, which lives in another process.
        For live captures this approach is necessary to avoid packet loss;
        for offline playback it is helpful for testing.
        Because of the weirdness of pypcap, target can be *either* an
        interface name or a pcap filename! :/
        """

        try:
          if self.live: self.minimize_packet_loss()
        except:
          log.info(traceback.format_exc())
          print "Failed to set kernel buffers to minimize packet loss!!!"
          print "You may encounter problems as a result of this..."

        try:
          if platform.system() == 'Windows':
            use_shell = True
          else:
            use_shell = False
          try:
            mode = '-i' # interactive
            if not self.live:
              mode = '-f'
            sz = "-b%ld" % self.packets_in_buf
            log.debug("Launching packet collector: "+ `[self.sniffer_thread, mode, self.target]`)
            try:
              self.sniff = Popen(
                [self.sniffer_thread, mode, sz, self.target], shell=use_shell,
                stdout=PIPE, stdin=PIPE, stderr=PIPE
              )
            except:
              # try bin/FastCollector
              self.sniff = Popen(
                [self.sniffer_thread2, mode, sz, self.target], shell=use_shell,
                stdout=PIPE, stdin=PIPE, stderr=PIPE
              )
          except:
            log.error("failed to start %s", self.sniffer_thread)
            raise

          if self.live: self.prioritise_sniffer()

        except OSError:
          value = sys.exc_info()[1]
          if "No such file or directory" in value:
            print self.sniffer_thread, "does not appear to be in the PATH..."
            #print "(which is", os.environ["PATH"] + ")"
          else:
            raise
          self.parent.quit_event.set()
          print "exiting"
          print threading.enumerate()
          sys.exit(1)

        self.tmpfile = self.get_tempfile_name()
        log.debug("Opening tempfile %s" % self.tmpfile)
        self.file = open(self.tmpfile, "a+b")
        fd = self.file.fileno()
        self.mem = mmap.mmap(fd, self.buffer_size, access=mmap.ACCESS_WRITE)

        self.parent.config.pcap_datalink = self.get_pcap_datalink()
        log.debug("Got pcap_datalink: %s" % `self.parent.config.pcap_datalink`)

    def prioritise_sniffer(self):
        """
        OS-dependent code to ensure the sniffer runs with the highest
        available priority level.
        """
        if platform.system() == 'Windows':
          util.set_win32_priority(self.sniff.pid, 5)
        else:  # UNIX is nice
          try:
            n = Popen(["renice", "-19", `self.sniff.pid`], stdout=PIPE)
            log.debug(n.stdout.read())
          except OSError:
            log.error("Attempt to prioritise sniffer failed with" + `sys.exc_info[0:2]`)

    def get_tempfile_name(self):
      """
      Communicate with the sniffer process to obtain the name of the
      mmap()ed tempfile through which it will pass us packets.
      """
      log.debug("looking for temp file..")
      self.sniff.stdin.close()
      line = self.sniff.stdout.readline()
      while line == "":
        line = self.sniff.stdout.readline()
        rval = self.sniff.poll()
        if rval != None:              # The sniffer died
          msg = "packet collector exited with return code %d\n" % rval
          msg += self.sniff.stderr.read()
          raise SnifferError, msg

      words = line.split()
      if len(words) == 0 or words[0] != "Tempfile:":
        raise SnifferError("packet collector didn't print a Tempfile: line")
      return words[1]

    def get_pcap_datalink(self):
        """
        Communicate with sniffer process to obtain the output of
        pcap_datalink on the capture (this will later determine header
        length of link layer portion of the packet).
        """
        line = self.sniff.stdout.readline() # XXX This is for "Initialized..." line
        line = self.sniff.stdout.readline()
        while line == "":
            line = self.sniff.stdout.readline()
            rval = self.sniff.poll()
            if rval != None:              # The sniffer died
                msg = "packet collector exited with return code %d\n" % (rval)
                msg += self.sniff.stderr.read()
                raise SnifferError, msg
        words = line.split()
        if len(words) == 0 or words[0] != "pcap_datalink:":
            raise SnifferError,"packet collector didn't print a pcap_datalink: line"
        return int(words[1])

    def run(self):
        """ sit in a tight loop and collect packets
            in interactive mode, run until the parent thread exits.
            in offline mode, break when the dump file ends """

        # This only returns on an error condition...
        try:
          self.launch_collector()
          self.read_packets()

          # The obvious thing to do with exceptions here would be to set the
          # quit event and then "raise".  But if there's a context switch in
          # between those two lines, we might never see the exception.  So
          # print it before sending quit_event.
        except SnifferError,e:    # sniffer error; clean up on the way out
          self.cleanup()
          log.error(e)
          if "handle a packet of size" in repr(e):
            self.large_segment_message()
          self.parent.quit_event.set()
          sys.exit(1)
        except:                   # What kind of error is this?
          self.cleanup()
          log.error(traceback.format_exc())
          self.parent.quit_event.set()
          sys.exit(1)

        # notify parent thread that there's no more input
        self.done.set()
        self.cleanup()
        # close pipe from zcat for gzipped traces
        if self._zcat != None:
            self._zcat.wait()
            if self._zcat_pipe:
                os.unlink(self._zcat_pipe)
        # organise shutdown for the entire client process
        log.info("Goodbye...")
        self.parent.quit_event.set()

    def filename_magic(self, filename):
        """
        Do all our housekeeping for playbacks of stored pcaps.  Sometimes
        these are gzipped, so we try to use named pipes to decompress them
        for the pcap library code.  Returns a filename for a pcap library.
        """
        # pcapy can't deal with gzipped input
        if filename.endswith('.gz'):
            trim_gz = filename[:-3]

            # if there's an ungzipped copy in the same place, use it
            if os.path.exists(trim_gz):
                filename = trim_gz

            # XXX this isn't going to work on windows
            elif platform.system() == 'Windows':
                print "sorry, I don't know how to open gzipped traces on windows"
                print "please gunzip %s and try again\n" % (filename)
                raise NotImplementedError

            # use a named pipe if we got em
            else:
                tmp = tempfile.mktemp()
                posix.mkfifo(tmp)

                self._zcat = Popen('zcat "%s" > "%s"' % (filename, tmp), shell=True)
                self._zcat_pipe = tmp
                filename = tmp

        return filename

    def read_packets(self):
      """
      Read packets out of the mmap()ed buffer that PacketCollector.py
      is busy writing them into.
      """

      self.pos = 0
      count = 0

      if platform.system() == 'Windows':
        self.check_for_sniffer_error = self.check_for_sniffer_error_win32
        self.sniff.fd_stderr = self.sniff.stderr.fileno()
        self.sniff.os_stderr = msvcrt.get_osfhandle(self.sniff.fd_stderr)
      else:
        self.check_for_sniffer_error = self.check_for_sniffer_error_posix

      while True:
        packet = self.read_one_packet()
        if packet == None:
          return
        else:
          timestamp, data = packet

        count +=1
        if count % lag_check_frequency == 0:
          delta = time.time() - timestamp
          log.info("Capturing a packet currently takes us %lf seconds" % delta)
        self.process_packet(timestamp, data)
        if double_check:
          print "Pos is now 0x%x" % self.pos
          print Ether(data).summary()

        #if count % 10 == 0:
        #  # at this point we aren't interested in normal playback termination
        #  # from the sniffer (ie, half_done==True).  We just need to know
        #  # about urgent conditions like dropped packets in live deployment
        #  self.check_for_sniffer_error()

    def read_one_packet(self):
      "Read a single packet out of the mmaped buffer; return (timestamp, data)."
      self.lock.acquire()
      try:
        if not self.tmpfile: return
        valid = self.mem[self.pos]
        while valid == '\x00':
          if self.check_for_sniffer_error(): return     # playback complete
          time.sleep(poll_interval)
          valid = self.mem[self.pos]
          if double_check: sys.stdout.write(".")

        base_pos = self.pos
        self.pos += vs

        len = struct.unpack("H", self.mem[self.pos:self.pos+pls])[0]
        if double_check: print "Packet of length", len
        if len > 1514:
          print "Wild packet length %d at pos 0x%x" % (len, self.pos)
          self.parent.quit_event.set()
          sys.exit(1)
        self.pos +=pls + padding2_size

        timestamp = struct.unpack("d", self.mem[self.pos:self.pos+tss])[0]
        if timestamp > self.latest:
          self.latest = timestamp
        else:
          t = self.latest - timestamp

          # Tell the user and the server every time we see packets
          # more out-of-order than anything we've seen yet
          if t > self.max_timewarp:
            msg = "Timestamp monotonicity violated by up to %g seconds" % t
            self.max_timewarp = t
            if t > 0.0002:  # 200 microseconds
              log.error(msg)
              # XXX if this gets up into the millisecond range we should
              # probably switch to a hard error.
              self.parent.link.send_message("error-cont", [msg])
            else:
              log.debug(msg)

        if double_check: print "Timestamp", timestamp
        self.pos += tss

        data = self.mem[self.pos:self.pos+len]
        # mark this packet as read
        self.mem[base_pos] = "\x00"
        self.pos = (self.pos + packet_size) % self.buffer_size
        return (timestamp, data)
      finally:
        self.lock.release()

    def process_packet(self, timestamp, data):
      "Integrate this packet appropriately into Alice's data structures."
      try:
        packet = Packet.Packet(
          timestamp + self._skew,
          data,
          self.parent
        )
      except:
        msg = "Packet parser failed on this packet (raw hex):\n"
        msg += hexlify(data) + "\n"
        if have_scapy:
          msg += "Scapy analysis\n" + `Ether(data).summary()` + "\n"
        msg += "The error was:\n" + traceback.format_exc()
        log.warn(msg)
        self.parent.link.send_message("error-cont", [msg])
        # HELP I'M STUCK IN AN EXCEPTION HANDLER!
        return

      if dummy:
        return
      if packet.is_fragment(): # skip fragmented packets
          if not self.frag_warn:
              log.warn("We saw fragments! This may result in false drop reports.")
              self.frag_warn = True
          return
      self.parent.fm.batch_to_process.acquire()
      # notify reporter thread if this packet completes a batch
      if self.parent.fm.handle_packet(packet):
          self.parent.fm.batch_to_process.notify()
      self.parent.fm.batch_to_process.release()

    def check_for_sniffer_error_win32(self):
      """
      Windows doesn't have "select" for pipes, so we have to resort to dirty tricks
      adapted from http://mail.python.org/pipermail/python-list/2006-August/396531.html
      """
      if self.half_done:
        return True
      try:
        data, avail, _ = PeekNamedPipe(self.sniff.os_stderr, 0)
      except: # pipe closed, we must be done
        return self.report_sniffer_error()
      if avail != 0:
        return self.report_sniffer_error()
      return False

    def check_for_sniffer_error_posix(self):
      """
      Return False for no condition; -1 for an error and +1 for a healthy
      shutdown
      """
      if self.half_done:
        return True
      # XXX perhaps we should use some combination of os.open() flags
      # and read() here instead of select...
      s = select.select([self.sniff.stderr], [], [], 0)
      if s[0]:
        self.report_sniffer_error()
        return True
      return False

    def report_sniffer_error(self):
      """
      go ahead and read from the sniffer's stderr, there's something there
      """
      print "We have news from PacketCollector:"
      ret = self.sniff.wait()
      output= self.sniff.stderr.read()
      if output:
        msg = "Packet sniffer error\n" + output
        try:
          self.parent.link.send_message("error-bye", [msg])
        except:
          pass
        raise SnifferError, output
      else:
        print "(Normal termination)..."
        print self.sniff.stdout.read()
        self.half_done = True
        return True

    def cleanup(self):
      # Alice may call this code too, so avoid deadlocks
      self.lock.acquire()
      try:
        if self.kernel_tweaked:
          self.restore_kernel_state()
      except:
        log.error(traceback.format_exc())
      try:
        if not self.tmpfile:
          return
        self.mem.close()
        self.file.close()
        try:            # to kill FastCollector
          if platform.system() == "Windows":
            # According to http://code.activestate.com/recipes/347462/,
            # this is how we killa process on Windows
            win32api.TerminateProcess(int(self.sniff._handle), -1)
          else:
            os.kill(self.sniff.pid, signal.SIGTERM)
        except:
          pass          # it's probably already dead
        time.sleep(0.3) # give the FastCollector time to exit
        if self.packets_in_buf > 25000:   # the old/default value
          log.info("Closing packet capture thread...")
          # this takes longer with large buffers!
          time.sleep (0.5 * self.packets_in_buf / 25000.0)
        try:
          cmd = ["shred", "-n", "1", self.tmpfile]
          shred = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)
          assert shred.wait() == 0
        except:
          # XXX some platforms won't have shred.  Ideally we should use their
          # native RNGs here... also, is this a good mode for open()?
          if not self.tmpfile:
            log.error("tmpfile already gone")
            return
          f = open(self.tmpfile, 'w')
          blank_entry = "\x00" * self.entry_size
          for n in xrange(self.packets_in_buf):
            f.write(blank_entry)
          f.close()
        try:
          os.unlink(self.tmpfile)
        except OSError, e:
          if "No such file or directory" in e:
            pass
          else:
            raise
      finally:
        self.lock.release()
        self.tmpfile = None

    segment_message = \
"""This error is usually a results of having a feature called "TCP Segmentation
Offloading" or "Large Segment Offloading" enabled in your operating
system.  See http://en.wikipedia.org/wiki/Large_segment_offload for more info.
Try running Switzerland with Segmentation Offloading disabled.
"""

    def large_segment_message(self):
      "Tell the user about large segement offloading and how to disable it"
      msg = self.segment_message
      plat = platform.system()
      if plat == "Linux":
        msg += "On linux, you can disable it with the following command:\n"
        msg += "ethtool -K %s tso off" % self.parent.config.interface
      elif plat == "Darwin" or plat[-3:] == "BSD":
        msg += "On %s, you can disable it with the following command:\n" % plat
        msg += "ifconfig %s -tso" % self.parent.config.interface
      elif plat == "Windows":
        msg += "On Windows, you should be able to disable it using the advanced "
        msg += "settings for your network driver."
      else:
        msg += "Sorry, we can't yet tell you how to disable it on %s" % plat
      log.warn(msg)
import sys
import time
import os
import platform
import traceback
import threading
import socket as s
import logging
            import psyco
from switzerland.client import PacketListener
from switzerland.client import Cleaner
from switzerland.client import Reporter
from switzerland.client.AliceLink import AliceLink
from switzerland.client.FlowManager import FlowManager
from switzerland.client.AliceConfig import AliceConfig
from switzerland.client.TimeManager import TimeManager, UncertainTimeError
from switzerland.client.Tracerouter import Tracerouter
from switzerland.common.PcapLogger import PcapLogger
#!/usr/bin/env python

#logging.basicConfig(level=logging.DEBUG,
#                    format="[%(name)s]: %(message)s")

log = logging.getLogger('')
log.setLevel(logging.DEBUG)
startup_handler = logging.StreamHandler()
log.addHandler(startup_handler)

welcome_msg = \
"""--------------------------------------------------------------------------------
Welcome to Switzerland.
READ THE PRIVACY AND SECURITY SECTIONS OF THE README.txt!!

Once you've done that... this is a Version Zero alpha release.  It's sure to
break at some point.  If/when that happens, please let us know by email
(switzerland-devel@eff.org), IRC (#switzerland on irc.oftc.net) or bug report
( https://sourceforge.net/tracker/?func=browse&group_id=233013&atid=1088569 )
--------------------------------------------------------------------------------"""

class Alice:
    """
    Alice is the Switzerland client.
    config is an AliceConfig object saying how she should operate;
    linkobj is only changed in test cases
    """
    def __init__(self, config, linkobj=AliceLink):

        self.config = config

        #logging.basicConfig(level=config.log_level,
        #                    format="[%(name)s]: %(message)s")
        config.check()

        self.init_logging()

        self.params = {}
        if config.use_ntp:
            self.ntp_setup()

        # local address may be fake for testing
        self.fm = FlowManager(config, parent=self)
        self.quit_event = threading.Event()
        self.cleaner = Cleaner.Cleaner(self)

        self.listener = PacketListener.PacketListener(self)
        self.link = linkobj(self.quit_event, self, config)
        self.reporter = Reporter.Reporter(self)
Next line of code: