pejman_gh
Joined 5 years ago
Joined 5 years ago
No. Instead of linting or static analysis, we tell you when a recent code change is causing your software to behavior or perform differently when handling a particular test case. Like Cypress but for software workflows that don't have a web interface or are not easy to test end-to-end.
> My company can barely invest in 80% unit test coverage. We have engineers writing tests to game coverage with no assertions.
Organizations are already investing in this other type of test by assigning QA teams to write and run manually or using software test automation tools. Touca offers a developer-friendly alternative to these tools that helps orgs save costs. It's a shift-left testing solution that helps engineers get the same confidence that QA teams provide, during the development stage.
From my experience, teams resort to superficial coverage goals because higher-level tests or the test infra needed to continuously run them are too expensive and complicated to run. We wanted to fix that.
I wanted a name that is easy to type on the command-line (since that's the interface for many users), easy to pronounce, and somewhat meaningless in English. Touca was the best I could think of. :)
We were not competing with or replacing GitHub Actions so the flow that you described wouldn't change. If you were to use Touca, you would continue to run "cargo test/npm test/dotnet test/whatever" that are likely running unit tests and integration tests.
Touca tests are more like snapshot tests and property-based testing in that you wouldn't specify expected values. You could run them locally or on CI or on a dedicated machine.
You would write high-level regression tests using our SDKs in Python, TypeScript, C++, or Java that would let you capture values of variables and runtime of functions for any number of test cases. The SDKs would submit that information to the our remote server that would compare them against the baseline version, and visualize and report any differences.
If you like to learn more: https://touca.io/docs/basics/ And you can find some product highlights here: https://touca.io/blog/touca-v2/
Having said that, our v2 launch last month was on the front page for about a day: https://news.ycombinator.com/item?id=34959276 ...but it was too little too late.
Sadly, building a must-have developer tools product is incredibly difficult as users are often not the primary buyers. One trick that some founders use is to postpone launching for real until after they raise one or multiple rounds. This way, they won't be held accountable for growth and revenue numbers. But that's dangerous since they can build for years with confidence, not realizing the flaws in their product positioning.
Startups are just really hard...
The example in Readme is passing the inputs as a list. You could also use a lambda function (see https://touca.io/docs/sdk/testcases/#programmatic-testcase-d...). Alternatively, you can remove the list and pass the inputs as command-line arguments. You can even choose not to pass them at all, in which case the test runner will re-use the list of test cases in the baseline version.
Storing test results in Git and comparing them via external tools may make it easier for teams to get started with Touca. But we care a lot about the developer experience and think that having a remote server take care of storage, comparison, visualization, and reporting could make a lot of sense for teams building serious software.
Touca has several components that work great together but you can certainly pick and choose and don't have to use all of them. We like Touca to do as much work as possible including integration with CI so that tests run continuously. But that integration is also optional. If teams have a different CI than GitHub Actions, they could call `touca test` as a CI step instead of using our plugins.
Does my explanation address your concern?
Reviewing diffs using `git diff` or `git blame` is not part of our roadmap. In fact we want to replace that model so that we can provide more insights and better summaries than just showing the diff between two textual outputs. Currently, you can use the Touca CLI to capture test results and get feedback whether they are any different. But you'd need to use our web interface to see and manage the comparison results in details. We're working to expand the CLI so that you could see the diff between any two versions right in the terminal.