test: add toxiproxy in front of postgresql in withTmpDb

[mkleczek]

DISCLAIMER:
This commit was authored entirely by a human without the assistance of LLMs.

Having toxiproxy between PostgREST and PostgreSQL in tests opens possibilities of validating behavior of PostgREST under various network conditions: temporary network partitioning, delays etc.

This change adds toxiproxy startup and configuration to withTmpDb function in withTools.nix, so that all communication goes through it in tests.

There is a Haskell and Python client available so that it is possible to easily control network conditions from both HSpec and io tests.

It would be very useful (if not necessary) to properly test #4673

[steve-chavez]

Great initiative 👍

under various network conditions: temporary network partitioning, delays etc.

Not against using toxyproxy, but just thought I'd mention we do have a custom tool to simulate delays: https://github.com/robx/slocat. It's already used in some places:

postgrest/nix/tools/withTools.nix

Line 250 in 2edc44c

${slocat}/bin/slocat -delay "$delay" -src "$PGRST_SERVER_UNIX_SOCKET" -dst "$REAL_PGRST_SERVER_UNIX_SOCKET" &

postgrest/nix/tools/withTools.nix

Line 250 in 2edc44c

${slocat}/bin/slocat -delay "$delay" -src "$PGRST_SERVER_UNIX_SOCKET" -dst "$REAL_PGRST_SERVER_UNIX_SOCKET" &

If not too complicated, an alternative would be to develop a similar tool (in Go or maybe C).

[mkleczek]

Great initiative 👍

under various network conditions: temporary network partitioning, delays etc.

Not against using toxyproxy, but just thought I'd mention we do have a custom tool to simulate delays: https://github.com/robx/slocat. It's already used in some places:

Yeah, I'm aware of slocat. We need a more comprehensive tool though. Toxiproxy is basically standard nowadays and is available in Nix packages.

[wolfgangwalther]

Yeah, I'm aware of slocat. We need a more comprehensive tool though. Toxiproxy is basically standard nowadays and is available in Nix packages.

Can toxiproxy do the same thing and more as slocat? Can we replace slocat?

[mkleczek]

Yeah, I'm aware of slocat. We need a more comprehensive tool though. Toxiproxy is basically standard nowadays and is available in Nix packages.

Can toxiproxy do the same thing and more as slocat? Can we replace slocat?

Yeah, I think so: https://github.com/Shopify/toxiproxy?tab=readme-ov-file#toxics

Wanted to leave removal of slocat for another PR though.

[wolfgangwalther]

Wanted to leave removal of slocat for another PR though.

Personally, I think it'd be easier to track by first doing a straight replacement of slocat with toxiproxy, allowing us to do the same things with it - and then extending the use of toxiproxy to other usecases. That will prevent us from having a state with both tools in there, which might become messy (I have not looked at the code at all, yet).

[steve-chavez]

Personally, I think it'd be easier to track by first doing a straight replacement of slocat with toxiproxy, allowing us to do the same things with it - and then extending the use of toxiproxy to other usecases

I agree, that would also make the PR easier to review + allow us to get familiar with toxyproxy for an existing use case.

[mkleczek]

Personally, I think it'd be easier to track by first doing a straight replacement of slocat with toxiproxy, allowing us to do the same things with it - and then extending the use of toxiproxy to other usecases

I agree, that would also make the PR easier to review + allow us to get familiar with toxyproxy for an existing use case.

I am splitting this PR into multiple commits. The first one is 6c90615

Not sure if you prefer them to be separate PRs or splitting them into multiple commits in a single PR is enough.

[steve-chavez]

Not sure if you prefer them to be separate PRs or splitting them into multiple commits in a single PR is enough.

It's easier to review on different PRs 🙏