A novel facile method for on-site detection of substituted aromatic pollutants in water using thin layer chromatography (TLC) combined with surface-enhanced Raman spectroscopy (SERS) was explored. Various substituted aromatics in polluted water were separated by a convenient TLC protocol and then detected using a portable Raman spectrometer with the prepared silver colloids serving as SERS-active substrates. The effects of operating conditions on detection efficacy were evaluated, and the application of TLC-SERS to on-site detection of artificial and real-life samples of aromatics/polluted water was systematically investigated. It was shown that commercially available Si 60-F(254) TLC plates were suitable for separation and displayed low SERS background and good separation efficiency, 2 mM silver colloids, 20 mM NaCl (working as aggregating agent), 40 mW laser power, and 50 s intergration time were appropriate for the detection regime. Furthermore, qualitative and quantitative detection of most of substituted aromatic pollutants was found to be readily accomplished using the developed TLC-SERS technique, which compared well with GC-MS in terms of identification ability and detection accuracy, and a limit of detection (LOD) less than 0.2 ppm (even at ppb level for some analytes) could be achieved under optimal conditions. The results reveal that the presented convenient method could be used for the effective separation and detection of the substituted aromatic pollutants of water on site, thus reducing possible influences of sample transportation and contamination while shortening the overall analysis time for emergency and routine monitoring of the substituted aromatics/polluted water.