Dynamically Loading Assembly for MSBuild Task

For a task that has to analyze an assembly using reflection (in this case
a task to export the hbm.xml or the Sql from an assembly with Plain Old .NET Objects
using NHibernate.Attributes)
one has to be able to load and unload the assembly dynamically (without that, it is
impossible to "build" two times the solution , because the first build will "block" the
assembly with the annotated Plain Old .NET Objects)
So, first I tried with loading the assembly using the Reflection Only Context but it turned out that the code inside of the NHibernate.Mapping.Attributes.HbmSerializer uses the
constructors of the attributes, which can not be called, to examine custom attributes loaded in the reflection-only context, use the CustomAttributeData class.
Well... I wasn't going to recode the HbmSerializer to make it work with the new .NET 2.0 API (not for now) so I have only one option left... create a new AppDomain an make the analysis there... but doing that is proving to be harder than I originally thought