mirror of
https://github.com/wiremock/WireMock.Net.git
synced 2026-01-11 14:20:29 +01:00
External Mapping Storage #574
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @PG-RichT on GitHub (Feb 22, 2024).
Originally assigned to: @StefH on GitHub.
Hi there,
Are there any plans to support an external source for the Mappings data?
I ask because I've started an exploratory feature branch to support Cosmos DB and a colleague suggested getting in touch in case this is on your roadmap? If not, do you have any suggestions or requirements that could ease the merging of this code into your master repo at some point?
Here's my very quick stab at how it could be done: https://github.com/WireMock-Net/WireMock.Net/compare/master...PG-RichT:WireMock.Net:cosmos_backend?expand=1
Thanks,
Rich
@StefH commented on GitHub (Feb 22, 2024):
So instead of storing the Mappings in memory, you want to store the mappings in CosmosDB?
@PG-RichT commented on GitHub (Feb 22, 2024):
Exactly :) We deploy a Wiremock server for a large suite of integration tests so it would great to be able to scale up the server instances as we dynamically add scenarios for specific tests.
@StefH commented on GitHub (Feb 22, 2024):
In that case, some ideas for the approach...
IMappingProviderinterface can be added to theWireMock.Net.AbstractionsprojectLegacyMappingProvidercan be renamed toDefaultMappingProviderand added toWireMock.NetprojectWireMock.Net.MappingProviders.Cosmosshould be createdPluginLoaderas defined in https://github.com/WireMock-Net/WireMock.Net/pull/1069@PG-RichT commented on GitHub (Feb 22, 2024):
Awesome, thanks for the pointers. Will keep you posted.
@PG-RichT commented on GitHub (Feb 22, 2024):
Can't move IMappingProvider to abstractions. It has a dependency on IMapping which itself has a dependency on a whole bunch of stuff in the Wiremock.net project.
@StefH commented on GitHub (Feb 22, 2024):
Good point.
In that case keep it in the main project.
@PG-RichT commented on GitHub (Feb 22, 2024):
Hmmm looking more and more like this won't be suitable for everyone; The Azure Cosmos SDK doesn't support 4.5.2 so I'll need to drop support for that.
@StefH commented on GitHub (Feb 22, 2024):
That is not issue if
WireMock.Net.MappingProviders.Cosmosis a separate project which does support the frameworks which are supported by Cosmos SDK.The same is for GraphQL and gRPC, that is only available for the newer frameworks.
@PG-RichT commented on GitHub (Feb 22, 2024):
Unfortunately due to the dependencies of IMapping, we can't have an external project reference the WireMock.net project for the Interface and then the Wiremock.net project reference a Cosmos project. It causes a circular dependency.
Implementing this is very low level, not really plugin. I think this is a fork forever...
@StefH commented on GitHub (Feb 22, 2024):
It is possible,
WireMock.Net.Matchers.CSharpCodeuses the same idea.The interface
ICSharpCodeMatcheris defined in main project and the implementationCSharpCodeMatcheris defined inWireMock.Net.Matchers.CSharpCode.Loading the type is done here:
2364866f97/src/WireMock.Net/Serialization/MatcherMapper.cs (L56)(Maybe PluginLoader is a wrong name, maybe better like TypeLoader or something...)
@PG-RichT commented on GitHub (Feb 22, 2024):
@StefH Ahhhhh ok, I'm starting to understand how you've done it now. Okay, I'll keep persevering 👍🏼
@StefH commented on GitHub (Feb 22, 2024):
Ok. Good to hear. In case you would like to build this logic and need an updated WireMock.Net, please tell me. Then I'll merge .
Also think on the naming:
Currently it's PluginLoader.Load and PluginLoader.LoadByFullName, and this last is the one you probably need.
So maybe:
or
Are better names?
@PG-RichT commented on GitHub (Feb 23, 2024):
Your new code will be essential so please go ahead. With regards to the name, TypeLoader has clearer intentions.
Thank you for your help with this, it is greatly appreciated.
@StefH commented on GitHub (Feb 23, 2024):
@PG-RichT
Code is merged.
@StefH commented on GitHub (Feb 23, 2024):
Would you like to create a PR for your new code? Then I can already add comments on the code while you develop?
@PG-RichT commented on GitHub (Feb 23, 2024):
@StefH Will do as soon I've resolved my current blocker - serializing the Interfaces used throughout a Mapping object in order to store them in the database.
@StefH commented on GitHub (Feb 23, 2024):
Yes, serializing the
IMapping/Mappingto JSON is not easy.What you could try is use the MappingConverter to convert a IMapping <> MappingModel (this is the same is used on the HTTP admin interface).
However, you will lose some functionality like the Func<> callbacks.
@StefH commented on GitHub (Feb 24, 2024):
@PG-RichT
I did some thinking.... 🤔
What you actually can do is just implement the
IFileSystemHandlerinterface.https://github.com/WireMock-Net/WireMock.Net/blob/master/src/WireMock.Net.Abstractions/Handlers/IFileSystemHandler.cs
This interface is used to read and write all the files needed by WireMock.Net , this also includes the mapping (MappingModel -JSON) files.
@PG-RichT commented on GitHub (Feb 24, 2024):
@StefH Thanks for that. I have started an approach (I don't know if it will work yet) that recognises an internal mapping i.e. one used by wiremock only, versus a user mapping.
Internal mappings will be stored in a concurrent dictionary like before, user mappings in cosmos.
I don't know how this will affect performance or if it will even work yet but let's see...
@StefH commented on GitHub (Feb 24, 2024):
There is technically no difference between a wiremock internal or a user mapping. These are all Mappings. So I think the best approach is : saving/reading the MappingModels as JSON in the CosmosDB using the IFileSystemHandler .
@PG-RichT commented on GitHub (Mar 1, 2024):
@StefH Apologies for a delay in my response. I have been assessing the implementation using the IFileSystemHandler and unfortunately I don't think it meets the requirements I would want out of backend storage.
Ideally the database of mappings would be called per request so any number of Wiremock instances could be spun up. If we go down the IFileSystemHandler route we introduce potential latency as each instance responds to an event telling it that 'the database has been updated, go refresh your in-memory mappings'. That latency might be tiny but the fact that it exists is enough to put me off.
Instead I have been looking at other infrastructure approaches to achieve our end-goal, sticky sessions appears to be simplest.
I wanted to thank you for your help so I could come to this conclusion though and for all the hard work and effort you put into maintaining this repo. Thank you.