Once a Maintainer: Maciej Mensfeld
Building an open source mentality by solving your own problems first.
Welcome to Once a Maintainer, where each week we highlight an OSS maintainer doing great work in the community and tell their story. This week we’re talking to Maciej Mensfeld, creator of the karafka gem and contributor to dry-rb and other Ruby projects. Karafka is a Ruby and Rails multi-threaded Kafka processing framework.
I first met Maciej in the Bundler Slack group, where he is a friendly and active community member. He dialed in for our chat from Kraków, Poland, where he lives and works.
How did you get into programming?
I think my story is quite similar to other people 20-25 years ago. I was just a kiddo. I think I was around 10 or 12 when we got a Famicon clone. It was called something different in Poland, and used mainly for games, you know, Mario and and stuff like that. This was my first encounter with microprocessors. Technically it is a computer, but I wouldn't call it a computer. It's just a device for games. I think like a year later I got my first real computer, Windows95, 500 megabytes of disk space, which was quite a lot at that time, CD drive and no internet.
The good and bad thing about those times in Poland is that technically speaking everything was open source, because there were no legal regulations around piracy and things like that. So you could consider any any software semi-open source, or at least, available. It was a fun time but I got bored quite fast with games. I didn't really get into extreme heavy gaming like some of my friends. I started being curious about how things work and how I could make a computer do what I want. Not just how I could play games.
And where in Poland are you from? What city?
I come from a really small city called Nowy Targ. I live in Kraków now. So you wouldn’t expect it from this small city. Once we were introduced to the binary system, and we had some exams and all the people expected the exam to be in, you know, binary, but then the teacher made it hexadecimal and he was like “it's the same, just change the base.” He was this type of a teacher.
One thing he taught me really well is how to teach myself. This is how I got into PHP. I got my first regular job building a CRM for one of the smaller banks here in Poland. And then one of my friends introduced me to, well, pre 1.0 Ruby on Rails. And here I am, still with Ruby, after so many years, it's just crazy.
What led to creating your own gem with karafka?
Karafka is quite old, it’s like eight years old now. What led me to creating it was just laziness. I used to work with a bunch of people and we needed an event-based system and Kafka was really, really fresh. It was this new thing that everyone wanted to try out. And so obviously, right, we wanted to find a use case for it, so I did find a use case. Then I noticed that we lacked any decent support in Ruby for it.
Once you start building, in my opinion, there are two types of libraries. There are libraries you add into your own as one of your dependencies and you just use them from whatever place you want. And then there are those dependencies, like Sidekiq, that need to deal with being a process, right? Handling signals, handling some sort of external TCP connections, instrumentation, logging, all the other weird stuff that is happening that you need to provide in order to be able to manage a process right? And I noticed that my colleagues at the company they struggled with it, so I had a choice. I could either explain to them how to do it. Or I could write them a wrapper that would just allow them to do their work. They wouldn't leave puma. They wouldn't leave the normal Rails space.
So I decided to write a wrapper because I'm a software engineer first and foremost. It was a rather simple solution that suited our needs. And then I noticed people got interested in it. And I like building stuff. I like sharing knowledge. So why not take this software that I built for a certain purpose, generalize it a bit, and expose it to a wider audience? And this is what I did. With any other open source projects that I contribute to, it’s the same - I'm just interested or I find a bug and then I want to understand it and then I want to fix it. This is how I got involved in the dry-rb ecosystem. There were things that were bugging me that I knew could be done better, so I just started committing. The way I got into the RubyGems security team was exactly the same way. There was a problem, I was interested in in finding a solution. I found a solution and joined the party.
So let's talk about your favorite Ruby package - you mentioned in your e-mail that it's actually from Ruby core, so why? Why that one?
So there is a hidden gem written by a guy named Seki. He's one of the Ruby core members, Seki-san. It's called drb, which is distributed Ruby. Basically if you take one hundred Ruby developers and ask them, “Do you know what drb/distributed Ruby is?” one or two will say yes. And if you ask them, “Did you ever use it commercially?” none of them will say yes, but it's an amazing piece of software that is shipped with Ruby.
There are a couple couple hidden gems in Ruby, but it's one of the ones that I love the most because it allows you to do seamless distributed computing, where you can spawn a whole bunch of processes that talk with each other without leaving the nice Ruby abstraction layer. Everything happens under the hood. It's also really well written. I don't know why it isn't popular or why it isn't popular outside of Japan, because I know that Seki wrote a book or two books about it in Japanese. They were never translated. And you know, I can only see the code samples.
Since it's standard, do you think maybe people just take it for granted and like you said, most people don't even know about it?
I think this may be part of the problem because it's not listed as one of the gems you're supposed to add to your technological stack. The second thing, I think it's just pretty advanced and you know, the majority of people that deal with it aren’t interested for many reasons in solving things the optimal way from a CPU management perspective and stuff like that - pretty low level stuff. It adds extra complexity because if you have more processes you need to do some resource management, you need to do some monitoring, you need to make sure they don't die, and if they die, you need to handle recoveries and stuff like that.
Still a beautiful piece of software and it encapsulates this magic, you know, and it makes it distributed. It's a really nice one. And he's a great guy by the way. He's super helpful, answers questions, problems.
How can we get more people into open source?
I think one of the things is C-level awareness - the executives need to be aware of how much they benefit from open source and they should be willing to give a bit of it back If you look at karafka, a couple of months ago I introduced a Pro version that gives you additional support and certain features, you know, like Sidekiq Pro. What is really funny is that all of the code of karafka, including the commercial features, are open source. That is, it’s all available for people to learn from. But in order to use it in commercial software you need to have a license, right? A different license than the GPL one, but the code is still viewable for people to learn- I keep everything at GitHub in a public repository. There's just a namespace called pro.
Some people are just shy and I've seen that happen. I would ask people, hey, you found this bug, just email the maintainer or open an issue. The majority of them would be happy to pick it up and look at it. We need to nurture people, especially junior developers. And maybe in order to get sustainable open source, some of it has to be somehow paid or there needs to be something getting back to developers. Mike Perham said a really good thing, that sometimes you just need to put a price on stuff, right? If you expect people to give you whatever you can, they will probably give you zero, or close to it. But if you put a price on something and there's a business justification, it's a win-win. You're able to sustain your open source and at the same time the company gets a benefit from it.
So it might be better if every piece of open source would become semi-commercial, but I think there can be balance somewhere in between. 98% of of karafka users use the free version and they're happy with it. That's all they need. But the 2%, it's not that much, but it's still something, right? You can pay for infrastructure and other things with that. A lot of people are not aware that open source doesn't necessarily mean zero cost for maintainers, aside from investing time. You quite often invest a certain amount of money into having the the proper infrastructure and paying for other things that are needed to do this type of work.
Who’s someone you think is doing really interesting work?
Peter Solnica also known as solnic, he lives in Kraków. He's one of the people that started all of the dry-rb libraries and he's always building something cool. He is doing a lot of interesting work around those libraries and Hanami.
Unfortunately for me I haven't had time to to play with Hanami a lot, but it's a great concept and technically built on really good principles. I would love to see it as a counterweight for Ruby on Rails. I don't think it's gonna happen, but I would love to see it happen. I think we need some framework diversity.
There are good things about having Rails and Rails only, and there are bad things about it. Obviously a good thing is if you have libraries like mine that need to interact or somehow connect with things like bigger frameworks - it’s easier if you have just one, right? You don't have to have maintenance of 20 different versions or five frameworks and you don't have to deal with this. And that's great, but I think that we we may lack some interesting engineering approaches that are not appreciated if you have 98 or 99% of the market taken by one framework and getting a new idea to spread becomes really hard. Having 20-25% of the market within a different framework would definitely help people that are interested in exploring off-road ideas.
Once a Maintainer is written by the team at Infield, an app that helps engineering teams keep their open source dependencies up-to-date.
To suggest a maintainer doing awesome work in the community, find us on Twitter at @infieldai. To subscribe, hit the button below: