hi h2

This idea was touched on in the sidux forums a while ago, but I don't think I explained it well enough to get your interest.

If the following code is put into smxi and the rm_warning function in it is called just before the actual dist-upgrade is run in the script, a user is warned if the dist-upgrade will remove any packages and he can't easily blindly just press 'y' to allow the dist-upgrade without seeing it is removing packages.



This would cover cases where dist-upgrades occur before warnings go up and take care of any instances of as yet to be discovered version mismatches that are not covered in smxi.

I'm not suggesting for a second that I think code as I'm presenting it is adequate for inclusion in smxi, I just thought that showing it might make the mechanization of what it does more evident.

The warning text I've written here is is just meant to give an idea and is specifically for sidux. As the functions as presented only cover dist-upgrades done with apt-get and not aptitude, sidux would probably get the most use out of it. But obviously it could easily be changed to test for and appropriately warn users running straight debian sid who use apt-get. The check that it does runs very quickly and is not noticeably obtrusive in the user experience.

I have one sidux install that I use smxi on that I d-u ~every two weeks and another that I d-u every day where I use my own simple scripts to update. I've been using this code in my script for about six months and it's saved me from my own carelessness several times. I inserted it in smxi and tried it out and it worked as it should have, although I'm not sure that how and where I called it was the best way (see second note below)

Thats about it. I find it really usefull, and if I'm not being dumb and missing something obvious, I think other users would too. But I'm not dealing with decisions on logical code mechanisms that can affect thousands of users, and I certainly respect your decision if you tell me to take a hike.

Just putting it out there for your consideration.

(I included the color variable tags in the code to try to make it consistant with your script in case you wanted to paste it into smxi and try it out. Of course, the colors only kick in when packages are to be removed, but at least trying it out would give you an idea of how fast it executes)

(How I tested it in smxi:

-- I knew transmission is out of sync and dist-upgrade would remove packages

-- Commented out test for transmission version mismatch in sm-lib-apt-tools

-- copied "warn_rm()" and "rm_warning()" functions into smxi above "install_dist_upgrade()"

-- smxi line 2813 : changed it to "rm_warning && $APT_TYPE $WITHOUT_RECOMMENDS $DU_UPGRADE 2>>$LOG_FILE 2>&1"

-- ran smxi -r

-- got the new "packages will be removed warning" right before actual dist-upgrade began

-- let d-u finish and let it remove trasmission, transmision-cli and transmission-gtk

-- let smxi finish

-- Then to test again:

-- uninstalled transmision-common -the package that caused the transmission packages to be removed

-- reinstalled transmission et all from testing

-- ran smxi -r again

-- same result, warning came up as expected
)

Sorry about the length of this. Pathetic thing is I was trying to keep it short.
Back to top

This one looks like a winner, I'll let the basic idea percolate for a while then see about tweaking outputs etc.

I can already see an instantly useful thing here, being able to log the full dist-upgrade/upgrade data no matter what else is used, I used to try to do that with tee, but tee is incredibly annoying to use, so I stopped, but this would solve that problem.

Don't worry about lengths, what I've learned in smxi is that you should use as much code/text to do the job as you need, not follow some idea about brevity that makes quality or clarity suffer.
Back to top

Really glad you think it's useful

Re your mentioning in the 'feature request' section that the warning message would have to be shortened to accommodate people with low resolution terminals: The text I used was only meant as an example, and to tell you the truth, it never occurred to me that not everyone uses framebuffer. But should it be decided that a similarly long message is desirable, it could be split into two sections with a 'press enter to continue' in the middle.
Back to top

I'm leaving this as a testing option for a bit, but if you remember to try smxi with -! 2 start options: smxi -! 2

-! 2 triggers testing flag 2.

This only runs if it's a sid system, if it's apt-get system, and currently, if it's using -! 2

I have an old install that I tested it on, and it worked well.


Back to top

hi h2

Running it with the test flags here worked very well and quickly. Haven't figured out a way to force a package remove situation so as to see it operate under that condition.

Very nice touch, adding the number and name of packages to be removed to the warning.

Also fascinating to me how in doing so it's now logical and easy for the function to trigger the warning on actually finding packages to remove instead of triggering it on finding ! "0 to remove."

(It's very cool to see what you did with this. I really enjoy scripting but I find I don't have any great aptitude for it.)

Speaking of aptitude, I saw you mention on the sidux forum that you haven't been able to find a way for aptitude to "answer no" in a similar way as the "--trivial-only" option does in apt-get. Have you ever considered using "expect"?

Before I new about the "--trivial-only" option in apt-get, I was using it to answer "no" to a dist-upgrade and it worked well and although I've never used aptitude, I imagine it could work there as well. The downside is it would add a dependency to smxi and I could see where that could be a problem. But it is a very small (316kb download) package and might be worth the trade-off.

Just a thought.
Back to top

expect is an idea, sure, I'll look into it. I in general want to avoid dependencies, but as you note, it's small. expect is odd though, but I'll keep that in mind to get both aptitude support and logging for upgrades.

I bounced around your method for a while, but it was too hard to get the packages removed using it, then I realized there were luckily unique search terms in the lines immediately before and after the remove list of packages, and sed rode to the rescue.

I'm going to place this slightly differently I think, but I haven't decided where exactly it will go, I'm leaning towards outputting this data along with the proceed with dist-upgrade, but then it's tempting to add an exit option, but I don't in general want most people to exit at this point, since as you noted in your text, packages getting removed is fairly normal in sid.

I'm bouncing around the idea of using some advanced regex to actually see if I can find the package that is removing the old one, in some cases, in the main package list, which would be VERY slick, but would be tricky to do, but in most cases, the packages have very similar names, off by a number or something in many cases.

I'll let the idea percolate for a bit, let me know if you have any further thoughts, this one isn't one that I'm going to launch immediately since it's a major change, but it is excellent information for people to get, I just want to make sure they get it in a form that doesn't make them falsely panic and exit.
Back to top

since I've tried expect before with little luck, I found by chance an easier, cleaner, and far smoother method to send aptitude the 'n' signal, something so obvious I would never have thought of it myself:



The trick is simple:

echo 'n' | aptitude dist-upgrade

that's it, plus I had to change a ffew of the search strings etc for aptitude, and add some cleanup so people don't look for packages like nfs-common{a}
Back to top

And now, if you look at the progression of this code in the function, you might start to understand just how smxi and modules have grown to be > 20,000 lines of code. This latest fixes a bug exposed by different user data than I had on my test boxes.



And with this, it gets closer to being feature complete, though the packages added feature I'll have to decide what to do with later.
Back to top

Three quick observations;

1) It's amazing (and a learning experience) to me when I look at this stuff how often it appears that one can write something that works in 20 lines, or can write something that ALWAYS works, for EVERYONE in 80.

2) In a million years I never would have come up with piping "echo 'n'" to apt-get/aptitude. If I somehow had considered it, I wouldn't have bothered trying it because I would have been sure that the 'n' would have been echoed and then would have just sat there waiting for someone to press <enter>. That is sooo cool.

3) I gotta quit avoiding diving into awk :)
Back to top

point one is the essence of all the scripts I do.

and it's why people like them, since they tend to work for them. Sadly, this idea, of making robust scalable scripts, is not very common in scripting, at least it's not the norm. That's a problem long term, because not only do the short ones tend to be very specific, they also tend to not be very robust, maintanable, or even readable.

functions, structure, abstraction from concrete to global/universal, every time I use these methods, my scripts are better, stronger, more robust, and more maintainable. Every time I get lazy ( a false laziness, by the way, since I always have to redo it to be robust anyway some time down the road), I end up having to spend more time than I saved making it right again later.

Point 2 is true, that's what I said too when a guy online told me that method, I also would have never expected a piped echo of n (remember though, that default echo includes a line ender at the end, ie, an enter) to trigger the n response in anything.

point 3: unlike bash or perl, awk is simple, clean, reasonably sane and rational, and, most important, consistent and logical.

trash80 was also doing some execution time testing on inxi, and discovered that embedding awk for some cases is also faster than trying to hack it out in bash.

bash, like perl, will I believe always remain a language that could have been good had the initial authors resisted the temptation to allow x different ways to do the same exact thing, and had they disallowed all obscure, write only type syntaxes that lead to far too terse code to ever maintain or even read by normal people, and, most important, had they actually treated it as a real language in the first place, ie, working multidimensional arrays that aren't lame hacks, functions that can return anything, thus preventing that ridiculous need to launch a subshell (which is a pain to debug the output of, or error handle) every time you want to get a non numeric, or > 255, return value (ie: fred=$( some_function ) ).

Sadly, my conclusion is now that for some reason, people who developed bash and perl both couldn't type at all, thus feeling that saving a few keystrokes now and then actually was worth sacrificing the readability of their languages long term, and suffered from insisting on using antiquated tools like vi, which in my opinion, by itself, makes code worse on average because it's just not easy to handle large code blocks with it, thus, again, encouraging bad practice of using shortest, most terse code possible, then proclaiming that unreadable mess 'good'.

That's my take after 3 years on these things. awk for some reason escaped this nonsense and actually is fairly reasonable, predictable, logical, and intuitive. I'm not sure how that happened given what happened with perl and bash, lol... but it seems to have escaped the worst tendencies.

But bash is the language I'm stuck with on these scripts, it's on all the target machines, it's relatively easy to handle once you learn to work around it's failings, and of course, there's no way I'm going to rewrite this junk anyway, and face a whole new set of bugs and issues a new language would create.
Back to top