Vista - Weird Symbol Problem ...

I've written a solution that supports a plug-in architecture. I load my
plug-ins in a separate app domain like Microsoft suggest for many reasons.

This is the code I use to load my plug-ins into my solution:

private byte[] Function1(string file)
{
byte[] buf = null;
using (FileStream fs = new FileStream(file, FileMode.Open))
{
buf = new byte[(int) fs.Length];
fs.Read(buf, 0, buf.Length);
}
return buf;
}

private void LoadFunc()
{
....
byte[] assemblyBytes = Function1("myAssemblyName.dll");
Assembly a = myPlugInAppDomain.Load(assemblyBytes);
....
}

All of the above code works great, and I'm not sure if it's related to my
issue, but I feel like I should let everyone know anyway just in case it is
related. So here's the issue:

My plugin gets loaded fine into the solution, but I can't debug my plugin.
I can successfully debug other dlls that are referenced by my plugin, but
not the plugin itself. The symbols aren't loaded.

The console app that I have basically loads the plugin from whatever dir it
is running out of. In this case, \solutionDir\projA\bin\Debug. I've copied
my plugin assembly from \solutionDir\projB\bin\Debug to
\solutionDir\projA\bin\Debug to get it loaded. I've also copied the pdb
that gets created into that directory as well.

The debugger (VS2008) tells me the following when I look at the symbol load
information:
Symbol search wasn't done for this module. Check if debug information was
generated when the module was built. Well, I'm pretty sure that the debug
information was built correctly. I've also tried doing the "Load Symbols
From"->"Symbol Path" option in the debug modules window, and selecting the
original \solutionDir\projB\Debug\ dir where my symbol file is located, but
VS2008 does not seem to like the pdb, even though I know it's the correct
pdb.

Does this have anything to do with the fact that I loaded the module from
memory rather than disk, or that it was loaded into a separate app domain?

Any help would be greatly appreciated. Oh yeah, I've also tried debugging
it by attaching to the console process, but that mad the debugger sad too.

Thanks,
-- John