starred/WireMock.Net-wiremock
1
0
Fork 0
mirror of https://github.com/wiremock/WireMock.Net.git synced 2026-01-11 22:30:41 +01:00
Code Issues 32 Packages Projects Releases 10 Wiki Activity

Sometimes returns status code 0 in unit tests with xunit test fixture (flaky test) #278

New Issue
Closed
opened 2025-12-29 15:19:34 +01:00 by adam · 8 comments
adam commented 2025-12-29 15:19:34 +01:00
Owner
Copy Link

Originally created by @erzki on GitHub (Jun 15, 2020).

Describe the bug

The problems is that sometimes our some unit tests that uses our WireMockServer fails by getting status code 0 in response from the mock server.
This happens in less then 1% of the test runs.

Here is from our log: System.Exception : Call /item/public_token/exchange failed with content and status code 0

We are spinning up a WireMock server in xUnit tests. We are doing it in a test fixture like this:

public class MockServerFixture : IDisposable
    {
        public WireMockServer MockServer { get; set; }

        public MockServerFixture()
        {
            MockServer = WireMockServer.Start();
        }

        public void Dispose()
        {
            MockServer.Dispose();
        }
    }

We have multiple tests using the same test fixture.

public class ServiceTest : IClassFixture<MockServerFixture>
    {
        private readonly MockServerFixture fixture;

        public ServiceTest(MockServerFixture fixture)
        {
            this.fixture = fixture;
        }

        [Fact]
        // Test goes here

Expected behavior:

The test should return the right result from the mock server, not just status code 0.

Test to reproduce

Very hard to reproduce because it's a flaky behavior.

Other related info

We started using test fixtures to not have to spin up the mock server in every single unit test which took about 1-2 seconds for every test. When having 1000+ tests (not all starting the mock server though) this makes a huge difference. So we think this flakyness started when running the tests with a test fixture.
The tests are not run in parrallel.

So the question is if anyone has an idea of what can be wrong here?

Originally created by @erzki on GitHub (Jun 15, 2020). ### Describe the bug The problems is that sometimes our some unit tests that uses our WireMockServer fails by getting status code `0` in response from the mock server. This happens in less then 1% of the test runs. Here is from our log: `System.Exception : Call /item/public_token/exchange failed with content and status code 0` We are spinning up a WireMock server in xUnit tests. We are doing it in a test fixture like this: ``` cs public class MockServerFixture : IDisposable { public WireMockServer MockServer { get; set; } public MockServerFixture() { MockServer = WireMockServer.Start(); } public void Dispose() { MockServer.Dispose(); } } ``` We have multiple tests using the same test fixture. ``` cs public class ServiceTest : IClassFixture<MockServerFixture> { private readonly MockServerFixture fixture; public ServiceTest(MockServerFixture fixture) { this.fixture = fixture; } [Fact] // Test goes here ``` ### Expected behavior: The test should return the right result from the mock server, not just status code `0`. ### Test to reproduce Very hard to reproduce because it's a flaky behavior. ### Other related info We started using test fixtures to not have to spin up the mock server in every single unit test which took about 1-2 seconds for every test. When having 1000+ tests (not all starting the mock server though) this makes a huge difference. So we think this flakyness started when running the tests with a test fixture. The tests are not run in parrallel. So the question is if anyone has an idea of what can be wrong here?
adam added the bug label 2025-12-29 15:19:34 +01:00
adam closed this issue 2025-12-29 15:19:34 +01:00
adam commented 2025-12-29 15:19:35 +01:00
Author
Owner
Copy Link

@StefH commented on GitHub (Jun 15, 2020):

@erzki
If possible, can you please create a project which uses the same fixture setup and maybe 1 unit-test which has a TestCaseSource which uses a for-loop to run the test 100 or 500 times?

And what framework (.net core or?) do you use for your unittests?

@StefH commented on GitHub (Jun 15, 2020): @erzki If possible, can you please create a project which uses the same fixture setup and maybe 1 unit-test which has a TestCaseSource which uses a for-loop to run the test 100 or 500 times? And what framework (.net core or?) do you use for your unittests?
adam commented 2025-12-29 15:19:35 +01:00
Author
Owner
Copy Link

@StefH commented on GitHub (Aug 1, 2020):

@erzki

  • what framework (.net core or?) do you use for your unittests?

  • A possible solution in WireMock could be to make the request handling forced synchronous (via a settings property), what do you think?

@StefH commented on GitHub (Aug 1, 2020): @erzki - what framework (.net core or?) do you use for your unittests? - A possible solution in WireMock could be to make the request handling forced synchronous (via a settings property), what do you think?
adam commented 2025-12-29 15:19:36 +01:00
Author
Owner
Copy Link

@erzki commented on GitHub (Aug 1, 2020):

Do you have any documentation om how to rum requests synchronously?

We are running the tests on .Net Framework still.

Den lör 1 aug. 2020 11:13Stef Heyenrath notifications@github.com skrev:

@erzki https://github.com/erzki

what framework (.net core or?) do you use for your unittests?

A possible solution in WireMock could be to make the request handling
forced synchronous (via a settings property), what do you think?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/WireMock-Net/WireMock.Net/issues/478#issuecomment-667500638,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMIMEIPUOJ4UEF5KDAAA33R6PMCDANCNFSM4N53QKPQ
.

@erzki commented on GitHub (Aug 1, 2020): Do you have any documentation om how to rum requests synchronously? We are running the tests on .Net Framework still. Den lör 1 aug. 2020 11:13Stef Heyenrath <notifications@github.com> skrev: > @erzki <https://github.com/erzki> > > - > > what framework (.net core or?) do you use for your unittests? > - > > A possible solution in WireMock could be to make the request handling > forced synchronous (via a settings property), what do you think? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/WireMock-Net/WireMock.Net/issues/478#issuecomment-667500638>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAMIMEIPUOJ4UEF5KDAAA33R6PMCDANCNFSM4N53QKPQ> > . >
adam commented 2025-12-29 15:19:36 +01:00
Author
Owner
Copy Link

@StefH commented on GitHub (Aug 1, 2020):

@erzki
Running WireMock.net synchronously is not yet possible, but I think this could be a solution for your issue. I'll take a look at the code and see if it can be changed.

@StefH commented on GitHub (Aug 1, 2020): @erzki Running WireMock.net synchronously is not yet possible, but I think this could be a solution for your issue. I'll take a look at the code and see if it can be changed.
adam commented 2025-12-29 15:19:36 +01:00
Author
Owner
Copy Link

@StefH commented on GitHub (Aug 1, 2020):

@erzki
I've added a property to the settings: HandleRequestsSynchronously.
You can try setting this to true.

You need a preview version from MyGet to test this:

@StefH commented on GitHub (Aug 1, 2020): @erzki I've added a property to the settings: `HandleRequestsSynchronously`. You can try setting this to true. You need a preview version from MyGet to test this: - WireMock.Net.1.2.17-ci-13663.nupkg - https://github.com/WireMock-Net/WireMock.Net/wiki/MyGet-preview-versions
adam commented 2025-12-29 15:19:36 +01:00
Author
Owner
Copy Link

@StefH commented on GitHub (Aug 6, 2020):

@erzki did you have time to check this?

@StefH commented on GitHub (Aug 6, 2020): @erzki did you have time to check this?
adam commented 2025-12-29 15:19:37 +01:00
Author
Owner
Copy Link

@erzki commented on GitHub (Aug 6, 2020):

Not really. I will try to test it but we have another approach in the plan.
To not use Wiremock in the unit tests. Instead some Http faker that doesn't
do real http requests.

Den tors 6 aug. 2020 20:18Stef Heyenrath notifications@github.com skrev:

@erzki https://github.com/erzki did you have time to check this?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/WireMock-Net/WireMock.Net/issues/478#issuecomment-670095254,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMIMEJ6TSZKIR5EEPQ6WX3R7LXWTANCNFSM4N53QKPQ
.

@erzki commented on GitHub (Aug 6, 2020): Not really. I will try to test it but we have another approach in the plan. To not use Wiremock in the unit tests. Instead some Http faker that doesn't do real http requests. Den tors 6 aug. 2020 20:18Stef Heyenrath <notifications@github.com> skrev: > @erzki <https://github.com/erzki> did you have time to check this? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/WireMock-Net/WireMock.Net/issues/478#issuecomment-670095254>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAMIMEJ6TSZKIR5EEPQ6WX3R7LXWTANCNFSM4N53QKPQ> > . >
adam commented 2025-12-29 15:19:37 +01:00
Author
Owner
Copy Link

@StefH commented on GitHub (Aug 7, 2020):

@erzki
That's also an possibility.

For now I'll close this issue, and merge the code to master (https://github.com/WireMock-Net/WireMock.Net/pull/496).

In case you have questions in the future, just raise a new question.

@StefH commented on GitHub (Aug 7, 2020): @erzki That's also an possibility. For now I'll close this issue, and merge the code to master (https://github.com/WireMock-Net/WireMock.Net/pull/496). In case you have questions in the future, just raise a new question.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
doc
doc
duplicate
feature
feature
invalid
pull-request
Mirrored from GitHub Pull Request
 question
wontfix
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/WireMock.Net-wiremock#278
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?