Servilia's Mother is Livia

I have found very little about her dates of birth and death, but assume that she survived both husbands, as she would have been substantially younger than them.

Servilia's grandfather is M.Livius Drusus.
Cos 112, Censor 109. He is Gaius Gracchus' opponent.
He dies during the Censorship in 109AD.

Servilia's uncle is M.Livius Drusus the younger
Tribunus plebis: 91 BC.
He advocated the reform of the equestrian courts. He also supported the Itali's demands for citizenship. But made a lot of enemies (incl. his brothers-in-law) and was murdered while being tribune of the people.

(I can't find any children, so Livia as his closest relative at the time would have inherited from him at this point, she would also have inherited the role of the patrons of the Itali and would have handed this one on to her children. This is a non-negotiable obligation - if called upon her decendants should act on their behalf).

Livia's first husband and Servilia's father is  M.  Servilius Caepio

Quaestor urbanus in 100BC. After first supporting the senators, after two lawsuits changes sites and fights with the equestrians against his brother-in-law M.Livius Drusus. He gets killed in the social war in BC90.

Livia must have divorced M.Servilius Caepio and was married to Cato by 95BC (no idea if this is the origin or the fall-out between her first husband and Livius Drusus). Servilia is born ca. 100BC, so she would have been 8 when her step-father died, 9 when her uncle died and 10 when her father died).
(by this time Servilia and her brother would have inherited all his wealth)

Servilia had a brother: Quintus (?) Servilius Caepio
who was a military tribune in the war against Spartacus
and later a quaestor to Pompey (in 67 BC)
But he died in Thrace.

Servilia's son Brutus was adopted by her brother (possibly after the execution of M.Iunius Brutus in 77BC) and would have inherited all his wealth at this point (aged c. 20).

Livia's second husband is M. Porcius Cato  
was tribune of the plebs in 99, a great friend of Sulla, and had a major fall out with is brother-in-law, M. Livius Drusus. But died before him (91 BC)
 (Having married into the most conservative family in Rome Servilia would probably have married in manu. Which means she would have inherited as joint heir together with Servilia's  half-brother Cato Uticensis and his sister Porcia)

Livia has two children with Cato: Cato Uticensis (born in 95BC) and a sister PORCIA (died in 45BC).
She is married to L.Domitius Ahenobarbus (who dies in the real world in 48BC at Pharsalos). He is a stout ally of Cato Uticensis and hated Caesar, especially as Caesar 'stole' his consulate in 55BC (he got it in 54BC instead). The senate in 49BC made him Caesar's successor in Gaul etc. (but then the civil war started).

The child of Porcia and L.Domitius Ahenobarbus is Cn.Domitius Ahenobarbus the younger , who is involved with the Ides of March (surprised?). He is old enough to take part in the civil war in 49 and gets pardoned by Caesar.

The Servilii claim descent on immigrants to Rome during the Royal periods. The Servilii even claim to come from Alba Longa, making the family older than Rome
The Porcii Catones claim to be from Tusculum in Etruria.
the Domitii Ahenobarbi claim to be the conquerors of Gallia Narbonensis.

I can't find any information on who brought the children up after 90BC, I assume it is Livia and her brother, but that is not proven.

Servilia is married twice by 85BC she is married to
M.Iunius Brutus (tribunus plebis in 83BC), who founded the colony in CAPUA (which makes his descendants the patrons of Capua), killed after the surrender of Mutina by Pompey in 77BC.

At this point Brutus, Servilia's son (aged 7 or 8), would have inherited his father's wealth in 77BC.

Servilia marries then:
D.Iunius Silanus (cos. 62BC, dead in 60BC). The result are three daughters:
Iunia Maior, Wife of M.Aemilius Lepidus, the later Triumvir, in 49 friend of Caesar , and makes him dictator, in 48/47BC he is governor in Spain.dies in 13/12BC). There are by 43 several children, which get mentioned in Cicero's letter to Brutus.
Iunia Secunda, wife of P. Servilius Vatia Isauricus, cos. in 48BC and 41BC. originally a supporter of Cato, from 50BC a supporter of Caesar. Their daughter Servilia (which must be about 10 years old in 49BC) is in 42BC briefly engaged to Octavian, she married in c.36BC the son of M.Aemilius Lepidus
Iunia Tertia, Cassius' wife.  

After 60BC Servilia did not marry again, but became Caesar's mistress and died at some point after 43BC.

Sources: Kleine Pauly, Oxford Classical Dictionary, Lexikon der alten Welt, Some of Cicero's letters.