⭐ Changes to how Rainmeter is released

[jsmorley]

We have decided to make a significant change in how "releases" are done for Rainmeter.

There will no longer be any "beta" releases of the software.
There will no longer be any "final" releases of the software.
There will just be a single "release".

Why?

What we have found is that the beta releases weren't really working as "beta" releases. Due to the long interval between final releases, the beta's were just being used as if they were final, and skins were being widely publicly distributed. In fact, it's likely that for a long time, almost nobody who was creating skins was using anything other than the beta versions.

What this meant was that we were in effect locked into "backwards compatibility" with the beta releases as soon as they came out. This was never the intention for the betas, as one of the things they were intended for is to try out some fix or change or improvement, get feedback, and maybe decide to change things, even if it was a change that would "break" any skins created with it.

The end result was that we didn't have a good way to publicly test proposed changes, and also that there was just no reason to bother with doing a "final" release in a timely way. This caused the stupidly long interval between the "final" release of Rainmeter 4.3.1 and now.

So what we want to do is get back to an approach where we can get proposed changes out there, and have people test them out and provide feedback, while not getting trapped into a situation where those changes are being distributed in skins all over the place, and we don't have the appropriate flexibility to react to suggestions and bug reports.

So what now?

Going forward, there will no further "beta" or "final" versions of the software. There will only be "releases", which we will do more often. The goal is to do a release pretty much as soon as we are done with any change, it has been tested by the team and in a new Pre-Release Testing forum we will be setting up, and we are satisfied that it is acceptable to roll out. More on the new Pre-Release Testing forum area soon...

In the very near future, we will take the following steps:

• Do a "release" version of Rainmeter 4.5.0. This will be based on the code from the latest, and last, 4.4.0 beta version.
• Remove any reference to beta versions from the main Rainmeter site, and remove the beta version of the Rainmeter documentation.
• Add the new 4.5.0 release version of Rainmeter to the WinGet repository, and remove the beta version(s)

For skin developers, this has the advantage of no longer needing to convince skeptical users of your skins that they need to install a "beta" version of Rainmeter. There will be only one version. For the Rainmeter developers, this has the advantage of being able to try out and test changes without fearing making changes based on feedback we get.

[death.crafter]

In the very near future, we will take the following steps:

• Do a "release" version of Rainmeter 4.4.0. This will be based on the code from the latest, and last, beta version.
• Remove any reference to beta versions from the main Rainmeter site, and remove the beta version of the Rainmeter documentation.
• Add the new 4.4.0 release version of Rainmeter to the WinGet repository, and remove the beta version(s)

For skin developers, this has the advantage of no longer needing to convince skeptical users of your skins that they need to install a "beta" version of Rainmeter. There will be only one version. For the Rainmeter developers, this has the advantage of being able to try out and test changes without fearing making changes based on feedback we get.

An welcomed me change indeed.

Can't wait to become a Rainmeter Insider

[jsmorley]

An welcomed me change indeed.

Can't wait to become a Rainmeter Insider

Hehe... That's the idea. Soon'ish...

[Yincognito]

A welcomed change to make every version neither beta nor final, but I have my doubts regarding the "pre-release" forum section, as I don't think it will get the attention required to make it an efficient and comprehensive feedback tool. In my opinion, regular folks will skip the pre-release forum section, just as they do now with betas, so the feedback will most of the times be incomplete, just as it's the case with Microsoft's "insider" similar approach in regard to Windows (and part of the reason why major bugs survive in their "official" OS versions). The section will probably appeal to more technical users, but as everybody knows, things are only properly tested "in the wild", by the entire mass of available users. Hopefully I'm wrong with this and it will work out for Rainmeter though...

[jsmorley]

A welcomed change to make every version neither beta nor final, but I have my doubts regarding the "pre-release" forum section, as I don't think it will get the attention required to make it an efficient and comprehensive feedback tool. In my opinion, regular folks will skip the pre-release forum section, just as they do now with betas, so the feedback will most of the times be incomplete, just as it's the case with Microsoft's "insider" similar approach in regard to Windows (and part of the reason why major bugs survive in their "official" OS versions). The section will probably appeal to more technical users, but as everybody knows, things are only properly tested "in the wild", by the entire mass of available users. Hopefully I'm wrong with this and it will work out for Rainmeter though...

While I entirely agree with your statement that "most regular folks will skip the pre-release forum", I disagree with "just as they do now with betas". Given that the version folks use is often driven by the "minimum version" requirements in a .rmskin, I would argue that "most" folks are just using the betas, and that isn't a beta testing approach at all. There is an implied conundrum in your statement that you don't think this new approach "will get the attention required", while also saying that most folks are skipping the betas. If both ARE true, it's still not an argument against this new approach in my view.

We gave this a lot of thought, and here is how we came down:

1) The "beta" versions were not being treated as a "beta" in any useful way anymore. They were just in effect a "final", and that caused us to no longer be able to test in a way that allowed the full range of fixes and changes we might want, due to backwards compatibility. One could argue that this was not due to the idea of "beta" in and of itself, and more due to the paucity of actual "releases" over time, and that's on us. However, the fact is that in our view, almost nobody was staying on the "release" versions anymore, as almost all the newer skins being released required the "beta". There certainly are folks who are just adverse to ever using beta software, and we respect that, but that just meant they pretty much never got the benefit of nice changes made over time, or newer skins folks released. This situation wasn't sustainable.

2) All the useful "testing" and "feedback" is coming, and always has come, from the more active users on these forums. Probably the only time "in the wild" testing is important is when there is a change where the computer hardware can be a factor. This may or may not come up in the future, but if it does, we feel that it is just as effective to just address the problem and do a new "release". It certainly can't be any worse than the situation today.

The idea is that we still want to "test" changes, but in a way that doesn't encourage public skins to be distributed all over the place until things have actually been tested. The "beta" as it is today is just not working that way. We see the new approach as more of an "alpha" test. I'm not opposed to the idea of "insider", although we don't want to call it that. Everyone is welcome. We kinda like "Pre-Release", as that has the correct "don't use this publicly just yet" feel that we want.

We decided to do this with a forums-only approach, and not to release them on the main website, or in GitHub or in WinGet and all that, both to streamline the implementation, ("ok, try this.. does that fix it?") and to discourage having these pre-release versions getting wide use.

Truth be told, it will also encourage us to get releases out quickly as soon as we are happy with a change. We have some changes in the works, that we are not quite yet ready to talk about, that will make staying current with the latest release version of Rainmeter a lot easier.

We are very careful to rigorously test anything we change, before it ever sees the light of day. Now we fully accept that a couple of foxes testing out stuff they themselves wrote is not enough to be certain, as I have said, "first rule of software development and henhouses", and we want and need the additional expertise that you guys bring to this. Time will tell, but we are hopeful that a fairly prominent forum area will get enough interest that we can get useful feedback. If guys like you and the others on the forums that constantly push the boundaries of what Rainmeter can do will participate, I'm pretty confident this will work.

This has the additional benefit of allowing the feedback we get from you guys to be more than "does it work?", but giving voice to "should we even do it this way?" that the old beta approach didn't encourage.

As I said, time will tell.

[Yincognito]

While I agree with your statement that "most regular folks will skip the pre-release forum", I disagree with "just as they do now with betas". Given that the version folks use is often driven by the "minimum version" requirements in a .rmskin, I would argue that "most" folks are just using the betas, and that isn't a beta testing approach at all. There is an implied conundrum in your statement that you don't think this new approach "will get the attention required", while also saying that most folks are skipping the betas. If both ARE true, it's still not an argument against this new approach in my view.

Good point. However, by regular user I meant a user that has "nice" skins on his PC, that are not necessarily new (as in to force him to update by the minimum version requirements), e.g. Monstercat, HipHopium ones, etc. I've seen quite a few cases like that lately, and that's what I meant with most regular folks skipping the betas, since if they didn't come to the forum for help they would have probably stayed with the old "final" versions (which weren't even the latest final version, by the way).

We decided to do this with a forums-only approach, and not to do anything but actual releases on the main website, and in GitHub and in WinGet and all that, both to streamline the implementation, ("ok, try this.. does that fix it?") and to discourage having these pre-release versions getting wide use.

Ah, I see, some sort of "internal" version(s) before the released one, leading to a faster update cycle. I'm curious how this will work, striking the right balance between encouraging and discouraging this from various uses will be key, otherwise we could get either "alphas" that are not tested in sufficient numbers (because user wants the version to be "official" and work well) or lots of "alphas" being used instead of released versions (because user wants to test the new features and participate in the version evolution).

We are very careful to rigorously test anything we change, before it ever sees the light of day. Now we fully accept that a couple of foxes testing out stuff they themselves wrote is not enough to be certain, as I have said, "first rule of software development and henhouses", and we want and need the additional expertise that you guys bring to this. Time will tell, but we are hopeful that a fairly prominent forum area will get enough interest that we can get useful feedback. If guys like you and the others on the forums that constantly push the boundaries of what Rainmeter can do will participate, I'm pretty confident this will work.

This has the additional benefit of allowing the feedback we get from you guys to be more than "does it work?", but giving voice to "should we even do it this way?" that the old beta approach didn't encourage.

Oh, participation and constantly pushing the boundaries of what Rainmeter can do is guaranteed from my (and probably others') part, you can rest assured. I just hope it will all have the desired (amount of) impact to make it a positive thing, that's all. As you said, time will tell - better to have some doubts and eventually be pleasantly surprised than having no doubts and end up dissapointed. Oh, and giving voice to "should we even do it this way?" definitely sounds interesting. Looking forward to it.

[jsmorley]

We are perfectly happy, and will actively encourage, having as many folks as possible participate in the "pre-release" process. Might not be a ton of folks, but I still maintain that is at worst no different than it is today, and likely much better. It is our intention to make it "rule one" that you get and use and test with the pre-release versions as much as you like, but don't, don't "distribute" skins publicly made with it. Wait for the "release", it won't be far behind. We can't "stop" people from doing so, but it will be on them, not us, when we completely change or remove some new feature. Guess we could have Rainmeter just refuse to create .rmskin files when you are running a pre-release version, but nah, too heavy-handed, and what if that is what we are looking to test?

[SilverAzide]

Guess we could have Rainmeter just refuse to create .rmskin files when you are running a pre-release version, but nah, what if that is what we are looking to test?

The skin packager has an input textbox for the minimum required version. I don't know if there is any validation on this field. I haven't tried putting in something goofy like "851.1.1", but could a check be added -- without a lot of hassle since ROI is low -- so that you could not create a .rmskin where the minimum required version was a pre-release version?

[jsmorley]

The skin packager has an input textbox for the minimum required version. I don't know if there is any validation on this field. I haven't tried putting in something goofy like "851.1.1", but could a check be added -- without a lot of hassle since ROI is low -- so that you could not create a .rmskin where the minimum required version was a pre-release version?

I think if we ever had to do that, it would mean this exercise has gone disastrously wrong. Kabul Airport wrong...

I find it unlikely we would have a lot of public skins released with the pre-release, when the first thing you will see in the new forum area is an admonition not to do that.

serious.jpg

[Yincognito]

The skin packager has an input textbox for the minimum required version. I don't know if there is any validation on this field. I haven't tried putting in something goofy like "851.1.1", but could a check be added -- without a lot of hassle since ROI is low -- so that you could not create a .rmskin where the minimum required version was a pre-release version?

Why not include in the RMSKIN.ini from the package a field like:

Code: Select all

BuiltWithRainmeter=...theversiontheskinwasbuiltwith...

and then on install, warn the user that "this skin was built with a pre-release version of Rainmeter and some features might be altered compared to released versions blah blah blah" with the options of proceed further or cancel installation? That won't force anyone to do this or that (i.e. give control to the user), but at the same time make him aware of the "risk", so to speak...

P.S. The version string would have to have some character that would identify a pre-release version from a released one, i.e. 4.4.0.3508a or something, obviously.