Ultrasound imaging is a veterinarian standard procedure for the monitoring of ovarian structures in cattle. Recent studies, suggest that the number of antral follicles can give a cue of the future fertility of a specimen. Therefore, there has been a growing interest in counting the number of antral follicles at early stages in life. In the most typical procedure, the operator performs a trans-rectal ultrasound scan and counts the follicles on the live video that is seen in the ultrasound machine. This is a challenging task and requires highly trained experts that can reliably detect and count the follicles in a quick sweep of a few seconds. This work presents the integration of several signal processing techniques to the problem of automatically detecting follicles in ultrasound videos of bovine cattle ovaries. The approach starts from an ultrasound video that traverses the ovary from end to end. Putative follicle regions are detected on each frame with a cascade of boosted classifiers. In order to impose temporal coherence, the detections are tracked across the frames with multiple Kalman filters. The tracks are analyzed to separate follicle detections from other false detections. The method is tested on a phantom dataset of ovaries in gelatin with dissection ground truth. Results are promising and encourage further extension to in-vivo ultrasound videos.