Startup race condition with ContentTypeModelScanner/ModelSyncInitialization initialization

[nzsb]

Hi,

Sporadically an exception is thrown at startup, which once led to some kind of strange corrupted Cache problem with a promotion.

Using DbLocalizationProvider 6.5.3 (I know, cannot update to 7 yet)

Seems to me that ModelSyncInitialization runs before DbLocalizationProviderInitializer which configures the handler *with UseSqlServer and co.)
Using
[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
for DbLocalizationProviderInitializer.

There are no custom resources at the promotion of any kind if this might be relevant.
Is there any way to make the initialization safer?

The exception:

2024-01-10 01:00:16,228;[ 6];ERROR;EPiServer.DataAbstraction.RuntimeModel.Internal.ContentTypeModelScanner;-;-;-;-;Failed to eager compile type Commerce.Common.ContentTypes.Promotion.CouponCodeFreeItemPromotion DbLocalizationProvider.HandlerNotFoundException: Failed to find handler for DbLocalizationProvider.Queries.GetTranslation+Query. Make sure that you have invoked required registration method (like .UseSqlServer(), .etc..) at DbLocalizationProvider.TypeFactory.GetHandler(Type queryType) at DbLocalizationProvider.TypeFactory.GetQueryHandler[TWrapper,TResponse](Object request, Type wrapperType) at DbLocalizationProvider.IQueryExtensions.Execute[TResult](IQuery1 query)
at EPiServer.Framework.Localization.ProviderBasedLocalizationService.LoadString(String[] normalizedKey, String originalKey, CultureInfo culture)
at EPiServer.Framework.Localization.LocalizationService.TryGetStringByCulture(String originalKey, String[] normalizedKey, CultureInfo culture, CultureInfo fallbackCulture, String& localizedString)
at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, String fallback, CultureInfo culture)
at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, CultureInfo culture)
at EPiServer.Framework.Localization.LocalizationService.GetString(String resourceKey)
at EPiServer.Commerce.Marketing.DataAnnotations.DistinctListAttribute..ctor()
at System.RuntimeTypeHandle.CreateCaInstance(RuntimeType type, IRuntimeMethodInfo ctor)
at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimePropertyInfo property, RuntimeType caType)
at System.Attribute.InternalGetCustomAttributes(PropertyInfo element, Type type, Boolean inherit)
at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.IsPropertyLazy(PropertyMapping property)
at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GeneratePropertyGetter(MethodBuilder getMethod, PropertyMapping property, Boolean isPage)
at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateProperty(TypeBuilder typeBuilder, PropertyMapping property)
at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateType(Type contentType)
at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory)
at EPiServer.DataAbstraction.RuntimeModel.Internal.IlGeneratedContentDataActivator.Create(Type contentDataType)
at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentTypeModelScanner.<>c__DisplayClass27_0.b__2()
DbLocalizationProvider.HandlerNotFoundException: Failed to find handler for DbLocalizationProvider.Queries.GetTranslation+Query. Make sure that you have invoked required registration method (like .UseSqlServer(), .etc..)
at DbLocalizationProvider.TypeFactory.GetHandler(Type queryType)
at DbLocalizationProvider.TypeFactory.GetQueryHandler[TWrapper,TResponse](Object request, Type wrapperType)
at DbLocalizationProvider.IQueryExtensions.Execute[TResult](IQuery1 query) at EPiServer.Framework.Localization.ProviderBasedLocalizationService.LoadString(String[] normalizedKey, String originalKey, CultureInfo culture) at EPiServer.Framework.Localization.LocalizationService.TryGetStringByCulture(String originalKey, String[] normalizedKey, CultureInfo culture, CultureInfo fallbackCulture, String& localizedString) at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, String fallback, CultureInfo culture) at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, CultureInfo culture) at EPiServer.Framework.Localization.LocalizationService.GetString(String resourceKey) at EPiServer.Commerce.Marketing.DataAnnotations.DistinctListAttribute..ctor() at System.RuntimeTypeHandle.CreateCaInstance(RuntimeType type, IRuntimeMethodInfo ctor) at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent) at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimePropertyInfo property, RuntimeType caType) at System.Attribute.InternalGetCustomAttributes(PropertyInfo element, Type type, Boolean inherit) at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.IsPropertyLazy(PropertyMapping property) at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GeneratePropertyGetter(MethodBuilder getMethod, PropertyMapping property, Boolean isPage) at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateProperty(TypeBuilder typeBuilder, PropertyMapping property) at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateType(Type contentType) at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory) at EPiServer.DataAbstraction.RuntimeModel.Internal.IlGeneratedContentDataActivator.Create(Type contentDataType) at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentTypeModelScanner.<>c__DisplayClass27_0.<EagerCompileProxies>b__2()

This exception happens in production around 1% of the startups.

[valdisiljuconoks]

Hei,

Yes, this seems to be an issue with init module requiring something from the localization provider while that guy is standing the queue for its turn to initialize.

Yeah, balancing on module call order is interesting exercise. This for example is module dep snapshot for sync module :)

The funny part is that order of init modules is not guaranteed and there is no way to control this from module point of
view either.

One option I can think of (not sure if this will blend tho) - would be to setup localization provider inside a configurable module (not initializable module). Configurable modules have a higher priority in call sequence.

[nzsb]

Thanks, sounds like a good idea, will try that, but implementation needs time as well as testing it on the long run.
This chart explains... everything :-)

[nzsb]

Hi Valdis,

Tried the recommended approach with configurable module like this:

    [InitializableModule]
    [ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
    public class DbLocalizationProviderInitializer : IConfigurableModule 

But we are still getting sporadically initialization issues, also with built-in types, like:

2024-07-15 00:00:23,733;[  13];ERROR;EPiServer.DataAbstraction.RuntimeModel.Internal.ContentTypeModelScanner;-;-;-;-;Failed to eager compile type EPiServer.Commerce.Marketing.Promotions.BuyQuantityGetShippingDiscount
DbLocalizationProvider.HandlerNotFoundException: Failed to find handler for `DbLocalizationProvider.Queries.GetTranslation+Query`. Make sure that you have invoked required registration method (like .UseSqlServer(), .etc..)
   at DbLocalizationProvider.TypeFactory.GetHandler(Type queryType)
   at DbLocalizationProvider.TypeFactory.GetQueryHandler[TWrapper,TResponse](Object request, Type wrapperType)
   at DbLocalizationProvider.IQueryExtensions.Execute[TResult](IQuery`1 query)
   at EPiServer.Framework.Localization.ProviderBasedLocalizationService.LoadString(String[] normalizedKey, String originalKey, CultureInfo culture)
   at EPiServer.Framework.Localization.LocalizationService.TryGetStringByCulture(String originalKey, String[] normalizedKey, CultureInfo culture, CultureInfo fallbackCulture, String& localizedString)
   at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, String fallback, CultureInfo culture)
   at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, CultureInfo culture)
   at EPiServer.Framework.Localization.LocalizationService.GetString(String resourceKey)
   at EPiServer.Commerce.Marketing.DataAnnotations.DistinctListAttribute..ctor()
   at System.RuntimeTypeHandle.CreateCaInstance(RuntimeType type, IRuntimeMethodInfo ctor)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimePropertyInfo property, RuntimeType caType)
   at System.Attribute.InternalGetCustomAttributes(PropertyInfo element, Type type, Boolean inherit)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.IsPropertyLazy(PropertyMapping property)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateMakeReadonly(IEnumerable`1 properties, MethodInfo baseMethod, MethodBuilder overriddenMethod)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.OverrideMakeReadOnly(Type contentType, TypeBuilder typeBuilder, IEnumerable`1 propertiesToOverride)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateType(Type contentType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.IlGeneratedContentDataActivator.Create(Type contentDataType)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentTypeModelScanner.<>c__DisplayClass27_0.<EagerCompileProxies>b__2()
DbLocalizationProvider.HandlerNotFoundException: Failed to find handler for `DbLocalizationProvider.Queries.GetTranslation+Query`. Make sure that you have invoked required registration method (like .UseSqlServer(), .etc..)
   at DbLocalizationProvider.TypeFactory.GetHandler(Type queryType)
   at DbLocalizationProvider.TypeFactory.GetQueryHandler[TWrapper,TResponse](Object request, Type wrapperType)
   at DbLocalizationProvider.IQueryExtensions.Execute[TResult](IQuery`1 query)
   at EPiServer.Framework.Localization.ProviderBasedLocalizationService.LoadString(String[] normalizedKey, String originalKey, CultureInfo culture)
   at EPiServer.Framework.Localization.LocalizationService.TryGetStringByCulture(String originalKey, String[] normalizedKey, CultureInfo culture, CultureInfo fallbackCulture, String& localizedString)
   at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, String fallback, CultureInfo culture)
   at EPiServer.Framework.Localization.LocalizationService.GetStringByCulture(String resourceKey, FallbackBehaviors fallbackBehavior, CultureInfo culture)
   at EPiServer.Framework.Localization.LocalizationService.GetString(String resourceKey)
   at EPiServer.Commerce.Marketing.DataAnnotations.DistinctListAttribute..ctor()
   at System.RuntimeTypeHandle.CreateCaInstance(RuntimeType type, IRuntimeMethodInfo ctor)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimePropertyInfo property, RuntimeType caType)
   at System.Attribute.InternalGetCustomAttributes(PropertyInfo element, Type type, Boolean inherit)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.IsPropertyLazy(PropertyMapping property)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateMakeReadonly(IEnumerable`1 properties, MethodInfo baseMethod, MethodBuilder overriddenMethod)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.OverrideMakeReadOnly(Type contentType, TypeBuilder typeBuilder, IEnumerable`1 propertiesToOverride)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentModelILCompiler.GenerateType(Type contentType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.IlGeneratedContentDataActivator.Create(Type contentDataType)
   at EPiServer.DataAbstraction.RuntimeModel.Internal.ContentTypeModelScanner.<>c__DisplayClass27_0.<EagerCompileProxies>b__2()

Do you have any other idea? :-)

Still using DbLocalizationProvider 6.5.3 with Episerver CMS 11.37.1.

[valdisiljuconoks]

Hi,

Is there any other init module you could hook into to get initialization earlier?
Another approach (just idea) could be to ignore "uninitialized" state of the provider when called pretty early in the pipeline and the eventually sync resources when it had its turn and called by the framework correctly..

[nzsb]

Hi,

Thanks for your input, I will research on the ideas, needs some reverse engineering :-)