mirror of
https://github.com/wiremock/WireMock.Net.git
synced 2026-01-11 14:20:29 +01:00
Random test failures between WireMock.Net 1.0.3.1 and 1.0.3.2 #94
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @alexangas on GitHub (May 1, 2018).
Originally assigned to: @StefH on GitHub.
Hello,
I attempted to update the version of WireMock.Net used by our test project, which was previously on 1.0.2.x. Now when the tests are executed we find some unexpected behaviour. Based on the output from the tests, WireMock sometimes appears to be returning the incorrect response - possibly what has been defined for another test?
A small percentage of tests are affected each time. The tests affected are not predictable, although there are a couple that come up every time that also have the characteristic of having 5 request registrations, while other tests only have 1 or 2. A guess is that the issue occurs when there are several tests running at the same time.
I've been able to narrow down the problem to changes introduced in version 1.0.3.2. I can see some thread safety / concurrency fixes were made, but the cause does not stand out to me.
Please let me know if you need any other details or assistance in resolving this issue!
Thanks, Alex.
@StefH commented on GitHub (May 16, 2018):
A quick question; does the latest version also have this issue?
@alexangas commented on GitHub (May 21, 2018):
Just tested again, and yes version 1.0.3.17 has this issue. If you have any theories on what might be happening, please let me know and i'll try to investigate!
@StefH commented on GitHub (May 23, 2018):
Do you use random ports in each test for the wiremock server?
@alexangas commented on GitHub (May 24, 2018):
Yes - we just make use of whatever WireMock chooses. If nothing stands out as the cause, is there currently an automated load test? If not, perhaps I can help out with that.
@StefH commented on GitHub (May 26, 2018):
There are no automated load tests, just internal unit tests (~200).
1]
Could it be that the
FindFreeTcpPortfunction does not work as expected?I can make a change in that code and maybe you can test the project from that changed branch?
Or maybe add a logging which prints the port?
2]
Do you start a new server in each unit-test ?
3]
Do you do something with mappings?
Because maybe I broke this line?
693778659e (diff-4590a9c246abe0a8ac30d2bc3a7abdcdR53)@alexangas commented on GitHub (May 27, 2018):
I spent a couple of hours writing some pretty amazing unit tests in a separate project to test load. Unfortunately, I could not reproduce the problem.
However, I did find what appears to be a thread-safe issue (which may fix it). It turns out that ConcurrentDictionary is not always thread safe if members are accessed through interfaces.
So, looking only at the
MappingsConcurrentDictionary, and ruling out the Explicit Interface Implementations, there are some uses that may be causing a problem -.Where(),.ContainsKey(), and.FirstOrDefault(). I haven't looked at any other uses of concurrent types.While it is more correct to use interfaces, to get around Microsoft's poor API design here, it may be wise to explicitly set all thread-safe collections as their exact type, e.g. ConcurrentDictionary instead of IDictionary. Then it should not be possible to have this occur.
@StefH commented on GitHub (May 27, 2018):
Interesting article!
I've applied the fixed to the code, please see (and review if possible?) https://github.com/WireMock-Net/WireMock.Net/pull/144
@StefH commented on GitHub (May 28, 2018):
Closing