Rydberg atom-based receivers of modulated radio frequency (RF) fields are promising systems for measurements. These systems are self-calibrating, widely tunable, nearly transparent to RF fields, and can be electrically small. However, the instantaneous bandwidth of current Rydberg atom receivers is typically less than 1 MHz. Using two-photon electromagnetically induced transparency (EIT) to observe the 56D5/2 Rydberg state in cesium, we measure modulation sidebands on each tooth in a probe optical frequency comb that spans the D2 F=4-F'=5 transition resulting from transmission modulation of the probe beam. This transmission modulation occurs from changes in susceptibility of the room temperature cesium vapor as two RF fields impinge on the atoms. A strong RF local oscillator is resonant with the 56D-57P state and mixes with a weak RF signal field detuned from the RF LO by an intermediate frequency. Using a self-heterodyned electro-optic comb setup, we separate positive and negative sideband amplitudes and compare to an equivalent comb-free system. These data report EIT measurement with the comb system, local spectra around two comb teeth - one within and one outside the EIT line, and normalized minimum detectable RF signal field as a function of RF intermediate frequency used to evaluate the instantaneous bandwidth of the single frequency, positive sideband, and negative sideband datasets.