In short, in older equipment yes and just one of the reasons that all electronic equipment must be disposed of by experts such as Secure Data recycling.

Capacitors containing polychlorinated biphenyls (PCBs) – Historically PCBs were used extensively in electrical equipment such as capacitors and transformers.  However, their use in open applications was widely banned in 1972 and they have not been used in the manufacture of new equipment since 1986.  Plants that had been installed prior to 1986 were allowed to continue until the end of their working life. Thus it should be assumed that capacitors manufactured before 1976 contain PCBs. 

Secure Data Recycling can give you advice and support in developing the WEEE program within your organisation.