Concerning Julia's death and Caesar's love for his daughter :

Tacitus, Annals III,6 : Tears and the solace found in mourning were suitable enough for the first burst of grief; but now they must brace their hearts to endurance, as in former days the Divine Julius after the loss of his only daughter, and the Divine Augustus when he was bereft of his grandchildren, had thrust away their sorrow. 

Cicero, On the Consular Provinces, 35 : if he were anxious himself to return to his country, to his household gods, to that dignity which he sees in store for him in this city, to his most charming children...

Plutarch, Caesar XXIII,5-6 : In Gaul he found letters which were about to be sent across to him. They were from his friends in Rome, and advised him of his daughter's death; she died in child-birth at Pompey's house. Great was the grief of Pompey, and great the grief of Caesar, and their friends were greatly troubled too;

Other references (unfortunately, for these, I've no english translation) :

Seneca, Ad marciam de consolatione XIV,3
Cicero, Ad Quint. Frat. III 8.3;1.17;1.7.25
Cicero, Ad Fam. VII 9.1
Valerius Maximus, On the Memorable Doings and Sayings V 1.10

See also :

Monroe E. Deutsch, Women in Caesar's family, The Classical Journal Vol. 13 n° 7 (04-1918) pp. 502-514
Franklin H. Potter, Political Alliances by Marriage, The Classical Journal Vol.29 n° 9 (06-1934) pp.668-674 (this paper contains also an interesting discussion on the identity of  Julia's former fiancé).

In a few words :

Servilia had two brothers,  Q. Servilius Caepio the older and Q. Servilius Caepio the younger ; the latter lived with Servilia in Cato's family ; he was Cato's beloved half brother and died in 67 BC.

It is likely that Q. Servilius Caepio the older was Pompey's lieutenant when he fought against the pirates in 67 BC and against Mithridates in 65. He adopted Marcus Iunius Brutus in 59. Henceforth Brutus' official name was Quintus Caepio Brutus. The question is wether Julia was bethroted to Brutus or his adoptive father. Probably to Brutus because his adoptive father was a political nobody. Plus, an inscription from Delos states that he was already married to Hortensius' daughter.

The argument against such an identification derives from Suetonius' account of the rejected suitor's activity against Bibulus in the early part of 59, Brutus beeing closely acquainted with Bibulus, Ahenobarbus and Cato who were Caesar's bitterest ennemies (Caesar XXI At about the same time he took to wife Calpurnia, daughter of Lucius Piso, who was to succeed him in the consulship, and affianced his own daughter Julia to Gnaeus Pompeius, breaking a previous engagement with Servilius Caepio, although the latter had shortly before rendered him conspicuous service in his contest with Bibulus). On the other hand, Brutus was also Servilia's son and Caesar's interest was to attach to himself a rising young politician by offering him Julia's hand. Then later, he found more usefull to give Julia to Pompey. Caesar's mistreatment of Brutus could be one of his reasons to join the Pompeian party during the Civil War.