command line - How to list all installed packages from a specific category (component)

Is it possible to list all installed packages from a specific official repository component (Main, Restricted, Universe or Multiverse) using utilities like apt, apt-cache, aptitude?

I wrote a simple script for this purpose:

dpkg -l | grep ^ii | cut -f3 -d ' ' | while read -r pkg;
do
 status=`apt-cache show $pkg | grep -m1 "Section: multiverse"`
 if [ ! -z "$status" ] 
 then
  echo $pkg
 fi
done;

It works, but it's really slow cause it's checking all packages one by one. Running the time command for this script will produce:

real    1m16.797s
user    0m57.008s
sys     0m8.260s

I already tried aptitude search patterns, and dpkg-query formating, but it seems they don't have the proper column/schema to create a query for this purpose.

I also had a look at vrms script to find out how it's works, because it's really fast in finding contrib/non-free packages, it's seems that vrms script scans the whole /var/lib/dpkg/status file, looking for things like 'Section: (contrib | non-free | restricted | multiverse | partner )', so it wasn't helpful either, because not all packages have this section.