Plus Sized Lobsters

Yes, I think yaml’s biggest strength is also its built-in flaw: its flexibility. Yaml as a data structure is built to be so open-ended that it can be no surprise when every component written in Go and using Yaml as a data structure builds their spec in a slightly different way, even when performing the exact same functions.

That’s why I yearned for something like CUE and was elated to discover it. CUE provides the control that yaml by its very nature cannot enforce. I can create CUE that defines the yaml structure in general so anything my system builds is valid yaml. And I can create a constraint which builds off of that and defines the structure of a valid kubernetes manifest. Then, when I go to define the CUE that builds up a KubeVela app I can base its constraints on those k8s constraints and add only KubeVela-specific rules.

Then I have modules of other components that could be defined as KubeVela Applications on the cluster but I define their constraints agnostically and merge the constraint sets together to create the final yaml in proper KubeVela Application format. And if the component needs to talk to another component, I standardize the syntax of the shared function and then link that function up to whatever tool is currently in use for that purpose.

I think it’s a good point that overgeneralization can and does occur and my “one size fits all” approach might not actually fit all. But I’m hoping that if I finish this tool and shop it to a place that thinks it’s overkill, I can just have them tell me which parts they want generalized and define a function to export a subset of my CUE for their needs. And in that scenario, I would flip and become a big proponent of “Just General Enough”. Because then, they can have the streamlined fit-for-purpose system they desire and I can have the satisfaction of not having to do the same work over and over again.

But the my fear about going down that road is that it might be less of an export of a subset of code and more of building yet another system that can MAD-style generate my whole CUE system for whatever level of generalization I want. As you say, it just becomes another abstraction layer. Can’t say I’m quite ready to go that far 😅

Thanks for the info. When I searched MASD, it told me instead about MAD, so it’s good to know how they’re differentiated.

This whole idea comes from working in a shop where most of their DevSecOps practices were fantastic, but we were maintaining fleets of Helm charts (picture the same Helm override sent to lots of different places with slightly different configuration). The unique values for each deployment were buried “somewhere” in all of these very lengthy values.yaml override files. Basically had to did into thousands of lines of code whenever you didn’t know off-hand how a deployment was configured.

I think when you’re in the thick of a job, people tend to just do what gets the job done, even if it means you’re going to have to do it again in two weeks. We want to automate, but it becomes a battle between custom-fitting and generalization. With the tradeoff being that generalization takes a lot of time and effort to do correctly.

So, I think plenty of places are “kind of” at this level where they might use CUE to generalize but tend to modify the CUE for each use case individually. But many DevOps teams I suspect aren’t even using CUE, they’re still modifying raw yaml. I think of yaml like plumbing. It’s very important, but best not exposed for manual modification unless necessary. Mostly I just see CUE used to construct and deliver Helm/kubernetes on the cluster, in tools like KubeVela and Radius. This is great for overriding complex Helm manifests with a simple Application .yaml, but the missing niche I’m trying to fill is a tool that provides the connections between different tools and constrains the overall structure of a DevSecOps stack.

I’d imagine any company with a team who has solved this problem is keeping it proprietary since it represents a pretty big advantage at the moment. But I think it’s just as likely that a project like this requires such a heavy lift before seeing any gain that most businesses simply aren’t focusing on it.

I’ve never heard of this before, but you’re right that it sounds very much like what I’m doing. Thank you! Definitely going to research this topic thoroughly now to make sure I’m not reinventing the wheel.

Based on the sections in that link, I wondered if the MASD project was more geared toward the software dev side or devops. I asked Google and got this AI response:

“MAD” (Modern Application Development) services, often used in the context of software development, encompass a broader approach that includes DevOps principles and tools, focusing on rapid innovation and cloud-native architectures, rather than solely on systems development.

So (if accurate), it sounds like all the modernized automation of CI/CD, IaC, and GitOps that I know and love are already engaging in MAD philosophy. And what I’m doing is really just providing the last puzzle piece to fully automate stack architecting. I’m guessing the reason it doesn’t already exist is because a lot of the open source tools I’m relying on to do the heavy lifting inside kubernetes are themselves relatively new. So, hopefully this all means I’m not wasting my time lol

Yeah, I’ve been using it heavily. While someone without technical knowledge will surely allow AI to build a highly insecure app, people with more technological knowledge are going to propel things to a level where the less tech savvy will have fewer and fewer pitfalls to fall into.

For the past two months, I’ve been leveraging AI to build a CUE system that takes a user desire (e.g. “i want to deploy a system with an app that uses a database and a message queue” expressed as a short json) and converts a simple configuration file that unpacks into all the kubernetes manifests required to deploy the system they want to deploy.

I’m trying to be fully shift-left about it. So, even if the user’s configuration is as simple as my example, it should still use CUE templating to construct the files needed for a full DevSecOps stack - Ingress Controller, KEDA, some kind of logging such as ELK stack, vulnerability scanners, policy agents, etc. The idea is the every stack should at all times be created in a secure state. And extra CUE transformations ensure that you can split the deployment destinations in any type of way, local/onprem, any cloud provider, or any combination thereof.

The idea is that if I need to swap out a component, I just change one override in the config and the incoming component already knows how to connect to everything and do what the previous component was doing because I’ve already abstracted the component’s expected manifest fields using CUE. So, I’d be able to do something like changing my deployment from one cloud to another with a click of a button. Or build up a whole new fully secure stack for a custom purpose within a few minutes.

The idea is I could use this system to launch my own social media app, since I’ve been planning the ideal UX for many years. But whether or not that pans out, I can take my CUE system and put a web interface over it to turn it into a mostly automated PaaS. I figure I could undercut most PaaS companies and charge just a few percentage points above cost (using OpenCost to track the expenses). If we get to the point where we have a ton of novices creating apps with AI, I might be in a lucrative position if I have a PaaS that can quickly scale and provide automated secure back ends.

Of course, I intend on open sourcing the CUE once it’s developed enough to get things off the ground. I’d really love to make money from my creative ideas on a socialized media app that I create, am less excited about gatekeeping this kind of advancement.

Interested to know if anyone has done this type of project in the past. Definitely wouldn’t have been able to move at nearly this speed without AI.

Definitely fake, I ain’t ever heard of a dog

Yeah, I have a lot of outdated books too.

Edit: I guess people don’t enjoy book puns?

Yeah, if I were you I’d cut my losses and try to find another place. If you’re lucky enough to know this place has bugs while very little of your stuff has been exposed, I’d get out before the problem has taken hold in your life.

That said, there are ways to deal with infestations. Likely if it’s been a problem dating back years, there’s some place they retreat to that kick starts the population each time they’re exterminated. But in typical homes, steam treatments from professionals can eradicate the pests. Mark Rober made a pretty good video pushing back on some of the stigma:

https://www.youtube.com/watch?v=2JAOTJxYqh8

Good luck to you. I hope you end up in a good place after all this. Sucks to get this close to a stable living space only to be thwarted by invertebrates.

I think each title/post of the same content should be treated as its own top-level object in the comments section, so collapsing everything at the top level would show you all the posts and reposts from various communities.

On client side, you should be able to merge all the posts, to sort all top level comments together. But if you go to make a top level comment, you’ll need to be replying to a specific post from a specific community (selectable, but defaults to the title you were shown from outside the post).

From outside the post, I think it would be cool to be able to browse the various posts of the same content from different communities, seeing their titles, the name of the community/instance, the number of comments.

Just my initial thoughts. Mainly, I just think it’s cool that we’re talking about this issue at all because once we solve this kind of problem in all its forms and iterations, we could see some really cool decentralized communities start to coalesce. IMO, the next big step after this would be building systems a user could use to find instances and communities they’re not yet aware of.

Same here. I’ve been building a bootstrap script, and each time I test it, it tears down the whole cluster and starts from scratch, pulling all of the images again. Every time I hit the Docker pull limit after 10 - 12 hours of work, I treat that as my “that’s enough work for today” signal. I’m going to need to set up a caching system ASAP or the hours I work on this project are about to suddenly get a lot shorter.

I spent about 20 minutes today trying to get Copilot on Word to tell me how to disable Copilot on Word. Worth every penny.

I also find USB to be limiting when it comes to range. I can go about 50 feet with a nice thick HDMI with copper wiring, but any further than 20 feet on USB necessitates fiber optics. Not an issue for everyone, but something I have been running into.

Hell, I might be a CFO now and I’ve failed to notice! Hold on, let me check… No… No, I’ve just managed to burn a microwave dinner again.

Get out. No, don’t get your coat. Just go.

Freeze! This is a stick up.

These are the worst puns I’ve ever seen, bar none.

I choose to take your questions as rhetorical, as I think our points do align. I quite agree.

My feelings are mixed. Everything you are saying is true. LLMs, right now at least, are a huge waste of resources. It’s triggering us to move closer to fossil fuels when we should be moving away. Every time I step outside to a nice balmy day, I think, am I going to miss this in a few years’ time? In a few decades, am I going to envy my current self who can do dishes without worrying too much about how much water goes down the drain? Are the generations to come going to look at my occasional can of tuna with contempt and jealousy? Or will they even have the luxury of retrospection?

I understand what we have to lose and how little we are doing about it. But I have also grown up being subjugated inside a capitalist hellscape. And I’ve spent the past few days having ChatGPT help me set up a CI/CD pipeline and start coding some games I’ve wanted to make for years. It’s allowed me to take a few hours of free time and make progress that I expected would have taken a week. It doesn’t have that effect on every task, but when learning new software, it really feels like having someone knowledgeable sitting next to me to answer my questions and point me in the right direction.

GPT 3 was kind of a neat party trick - sounds kind of like a person, but a pretty dumb person. GPT 4 sounded smarter, but still couldn’t code for shit. The o1 model still makes mistakes, but it retains the thread of our conversation weeks after the fact and has put together some code that I would have struggled to do myself. Even if it loses more money than it makes right now, I can see the value in progressing development until we achieve AGI.

People have expressed hopes that AGI will solve a lot of the world’s problems. That it will know just what to do about climate change. That it will crack codes in our DNA and give us endless healthy life. I am doubtful that these dreams will come to fruition. At least not in the way people think. It might have the intelligence to tell us things that we should have already known. Like that we can’t get much better yields in scrubbing carbon from the air than nature itself and we should have reforested far more land than we currently are. And that immortality will take huge amounts of resources and will come at the expense of the health of the masses. More gain for the rich. More suffering for the poor. Business as usual.

But I think there is a window of time where we can be hopeful about what AI has to offer. And we may even be able to leverage it to solve a big piece of the income inequality puzzle.

If we make a social media app that is not designed for profit but instead for the good of the people, there are a lot of problems such an app could solve.

We could design it to seek out real (non-bot) contributors. It will always be an arms race trying to sort real humans from bots but that is no reason to give up. It is a reason to get as far ahead in the race as we possibly can. We should build an app that both recognizes when someone is very likely to be real and when they have also contributed to a cause.

Imagine an application that tracks creative innovation, such as the creation of a funny video or a new meme format. When someone makes an idea and it is popular, the AI model would determine how much of a given experience is improved by their idea and give them profit residuals based on their contribution. And the more ideas that get built on top of the original idea, the more the newer contributors are rewarded for their contributions.

Think about if people could design a farm from the ground up using a socialized app for collaboration. Someone could design a camera system to keep track of livestock wellbeing and to head off diseases. They could make AI-empowered systems to track livestock happiness and find ways of increasing quality of life. And creating more humane automated methods of turning crops and livestock into food ready to transport. Some people would focus on creating ideal distribution methods. Others would create stores or restaurants. Others might work on the people themselves, encouraging them to give new more climate friendly meal options a try. Investors would be paid their dues, but there would be no CEO or board of executives. The means of production would belong to the people.

When people talk about the potential of AI, that’s what I envision. If I can make some passive income with my games and apps, that’s the next project I’ll be diverting my time towards. Because this is a narrow window we have to make this happen. The technology is here, but barriers from climate change and income inequality are only going up. We can lament the fact that AI is currently not profitable and hurting the planet, or we can put more of that energy to use by taking the tools humanity has made and using them to dismantle the systems which made this timeline so intolerable to begin with. The only way to take the current system apart is to make a new one that outcompetes our old ways of life in every measurable way.

Hey, that’s longer than I can run.

It’s got your number.