clafferty

Declarative workflows is such a good idea, fantastic, and I love the AI first principles where pipeline creation and editing the pipeline can be done with AI too.

The declarative style keeps the workflow detail at a high enough level to iterate super quick - love that. More important to me is that it’s structured and seems like it would be more testable (I see validation in your docs).

Zooming in to the pipe/agent steps I can’t quite see if you can leverage MCP as a client and make tool calls? Can you confirm? If not what’s your solution for working with APIs in the middle of your pipeline?

Also a quick question, declarative workflows won’t solve the fact that LLMs output is always non deterministic, and so we can’t always be guaranteed the output from prior steps will be correct. What tools or techniques are you using/recommending to measure the reliability of the output from prior steps? I’m thinking of how might you measure at a step level to help you prioritise which prompts need refinements or optimisations? Is this a problem you expect to own in Pipex or one to be solved elsewhere?

Great job guys, your approach looks like the right way to solve this problem and add some reliability to this space. Thanks for sharing!

You’re going to need to make a few aggressive WAF Rules, pepper in some whitelisting rules and if you can, add rate limiting.

1. Block all unverified bots with a bot score of 1. This will still allow popular web crawlers but could be strict enough to block a curl request.

2. Use Manage Challenge for unverified bots with a bot score less than 30. This will silence most of the trouble making bots and provides a JavaScript (not necessarily Captcha) solution for users who are incorrectly scored.

3. Add rate limiting. Figure out a realistic access rate, double it and use that as a hard limit that will block traffic for an hour or day depending on your needs.

4. Add more sensitive rate limits and play with manage challenge rules. Use the simulate option before enabling any rate limits. You can add challenges here if you feel a limit might be affecting users too. Simulate for a few days before enabling

5. Review rate limits and firewall reports regularly and adjust. With any Managed Challenge rules make sure to check the percentage completed to see if you’re trapping real users. This number should be as close to 0 as possible. Repeat step 4.

You’ll want to get around your own blocking rules with some complimentary whitelisting rules.

Although it’s advised to lock down your origin server to prevent non Cloudflare traffic hitting your server you might not be able to do so easily, if you’ve got load balancers and other infra in your way that can’t be touched. Just make sure your root domain isn’t leaking your www IP address. You can use CNAME flattening and you should be alright.

The difficulty in these solutions is managing all the rules you can make. Things can quickly become too complicated to make changes easily. Keep it simple, have a few basic but aggressive blocking rules and revise your whitelist and rate limits regularly. Good luck

Nice work Jiga!

Congratulations Jiga for making it onto HN! So excited to see where you take this. Best of luck, Conor

Bravo m3o, this looks awesome!

Netlifys approach to static sites seemed like a no brainer since HTML is easy to host. They added lots of sugar around that to make it effortless and easy. So M3o seems like a great idea.

Micro on the other hand will need some buy in but I'll definitely consider it since we use Go already.

One concern is your pricing seems too reasonable (free)!

Did my bit to chip in. Some hope still!

This looks awesome, great job! One thing that will slow me down from using this is I've not settled on an ID or Access Management system. Being a small company, we occasionally need to grant system access to contractors or other dev teams. The problem is we don't want to grant the access too wide and specifying fine grained controls takes a lot of time.

Armon mentions Okta and Ping, does anyone have any recommendations in this space that would work for managing a small team with occasional on/off boarding of contractors?

Theres a bug in this when Math.random() === 1