New recording technologies are transforming neuroscience, allowing us to measure the spiking activity of thousands of neurons in freely behaving animals. These technologies offer exciting opportunities to link brain activity to behavioral output, but they also pose serious statistical challenges. Neural and behavioral data are noisy, high-dimensional time-series, and in experiments designed to study animals' natural behavior, there may not be a clear trial structure to offer repeated measurements. I will present my lab's work on unsupervised methods for discovering repeated structure in neural and behavioral time-series. The key ethological hypothesis behind our approach is that behavior is composed of sequences of discrete, stereotyped, and reusable actions, which we call behavioral "syllables." First, I will show how we are using probabilistic state space models to identify syllables in behavioral data and relate them to simultaneously recorded neural activity. Then, I will approach the same problem by searching for repeated structure in neural data instead. Recent experimental work suggests that certain regions of the brain encode behavioral syllables with sequences of spikes in a subset of neurons. I will show how Neyman-Scott processes — a class of doubly stochastic point processes — can find such sequences, even when they are embedded in high-dimensional neural recordings. Together, these lines of work demonstrate how advances in probabilistic modeling and inference enable new types of neuroscience experiments and provide insight into the link between brain activity and behavior.