-U Error 33: Error downloading file with curl: https://github.com/smxi/inxi/raw/master/inxi; Error: main branch
fm
Status: Curious
Joined: 01 Jan 2019
Posts: 5
Reply Quote
I have a bunch of different distros installed across a bunch of machines, the largest number of which are openSUSE, but also included are Fedora and Debian among others. I often get inxi -U failing with the subject messages. /etc/inxi.conf includes B_ALLOW_UPDATE=true and GLOBAL_COLOR_SCHEME=0 on all. Is this routine for others too?
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3895
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
Problem is on your end, not inxi's.

There's no such thing as an http web connection 'sometimes' not working (at least not on a consistent basis), but there is such a thing as your local connection sometimes failing. Over the years I've encountered end user network failures (more so with smxi/sgfxi, which fired off so many quick downloader request that they would often trip failures on the user local level), and the oddest thing always struck me, despite the only links on the network chain that are low quality consumer grade equipment being on the user's LAN, and possibly on their ISP's node equipment directly upstream from them, people almost always believe that such random failures would most logically be originating upstream from that, on the best networking equipment and servers in the world, lol.

To put this simply, the job of inxi's downloader logic is to tell you when the download has failed. It is apparently doing its job quite accurately, so it's up to you to debug that.

You can also use --dbg 1 to get more debugger data, but in your case, it's not going to matter since the download failed. It may or may not have more info to give, but it will give you the full command you can then run yourself outside of inxi to see what is going on with your local area network. My guess is usually a defective router in your LAN, out of ram (particularly likely if you use wifi for your connection, which eats router resources), or a defective local ISP node switch.

:: Code ::
sudo inxi -U --downloader curl --dbg 1
Starting inxi self updater.
Using curl as downloader.
Currently running inxi version number: 3.0.30
Current version patch number: 00
Current version release date: 2018-12-31
Updating inxi in /usr/local/bin using main branch as download source...
-------
Downloader Data:
curl   -y 4  -L -s  https://github.com/smxi/inxi/raw/master/inxi
Result: Success: stdout data not null.
Successfully updated to main branch version: 3.0.30
New main branch version patch number: 00
New main branch version release date: 2018-12-31
To run the new version, just start inxi again.
----------------------------------------

Starting download of man page file now.
Downloading Man page file...
-------
Downloader Data:
curl   -y 4   -L -s -o  /usr/local/share/man/man1/inxi.1 https://github.com/smxi/inxi/raw/master/inxi.1
Result: 1
Download successful. Compressing file...
Download and install of man page successful.
Check to make sure it works: man inxi


As an aside, I may remove the silent options from the doewnloaders if --dbg 1 is used, that would make sense, I'll see if that works maybe.

There's a corner case I encountered with a sys admin who belonged to a distro team I was in, and he could not grasp that it was HIS network that was causing connection problems, it was almost sad/comical to see him deny it could be his network, when thousands of other users daily were using the same program doing the same thing, and it was only his network that had issues. In his case, I suspect there was a firewall rule somewhere in a router that he didn't realize could trip those filters.

Diagnosing other people's network failures is sadly far outside of the scope of what inxi or I can offer help with though.
Back to top
fm
Status: Curious
Joined: 01 Jan 2019
Posts: 5
Reply Quote
Is there a way I can simply feed the update URL(s) to wget to instead of curl? When the curl failures occur here, nothing else evident is failing. My inxi updates are usually within a minute or so of other updates, such as zypper and apt do without making any network errors apparent. Maybe the only problem is broken curl, which I never use directly.

When I wrote "sometimes" what I meant is that on some installations (I have hundreds) it sometimes works, while on others not, as well as not working on first try, then running -G or anything else, then trying -U again. Sometimes the retry succeeds, other times the same curl failure repeats. The failures seem to be most common when the starting version is 3.0.29 or when 2.4.whatever originally came into Stretch via apt.
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3895
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
Ah, you may actually have a similar problem that the sys / network admin I mentioned at the bottom of my response had.

If it consistently doesn't work on some installations, that means it always fails, which is of course a lot easier to diagnose than it sometimes failing.

I'd look at firewall rules somewhere interfering with the curl request. In the previous case where a sys admin had this issue, my personal feeling was that there was a switch directly upstream from his system that was tripping the block, or firewalls on his network that were filtering outbound requests that matched certain behavioral patterns, but this was just a guess since the guy refused to accept that it was his system at fault.

inxi has the --downloader option which is made to force a specific downloader. Old inxi had I think a similar option though I dont' remember what it was.

It looks to me like you may have exactly the same issue the sys admin I dealt with years ago had, some type of configuration on his network that would block multiple wget firings but let things like apt requests through. We were never able to actually diagnose the real cause of this because he refused to admit that the problem was on his network, or directly upstream from his network, so it remained a mystery, but given you seem to be running a similar sized network, I'd bet there is some configuration somewhere that thinks repeated curl requests are 'bad'. Just a guess, as noted, we could never track down the cause, which was never smxi's fault, it was always dutifully reporting the failure to receive the desired packets when asked to get them.

To really track this down, you'd have to use maybe some type of packet analyzer to find out what is returning the failure codes, though it may not actually 'see' the cause.

I don't test much on curl beyond verifying it works, but it's unlikely it's going to fail in this way, unless there was a known bug in some releases that has been fixed, no way to say.

I realize that actually tracking down and diagnosing such failings is a real time suck, but I'd personally be very interested if you can figure out what is causing it.

Note that with smxi, my servers served all the files, so I knew that it wasn't on my end, in the case of inxi, it's github, and I can't say anything about how they handle cached file requests, but it would be super odd, and a massive bug, if their systems denied requests simply because they were curl, but not impossible
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3895
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
If you don't mind, I'll add a tweak to pinxi's --dbg 1 that removes the quiet options for wget/curl/fetch, that will let you see what is happening. Not positive that will work since inxi download goes to stdout, but I'll check and if it works, I'll commit that change.
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3895
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
pinxi (not inxi) version 3.0.30-5 now adds in removing the -s/-q quiet flags from fetch, wget, curl, which may or may not help diagnose failures.

I should have done that a long time ago, so that might help others in the future since the point of --dbg 1 is to diagnose downloader failures.

I just realized that --dbg is not documented in man or help, that's because those are only internal debuggers, which can change any time to do whatever, but actually --dbg 1 is always for downloaders so I'll document that one.

:: Code ::
pinxi -U --dbg 1 --downloader curl
Starting pinxi self updater.
Using curl as downloader.
Currently running pinxi version number: 3.0.30
Current version patch number: 05
Current version release date: 2019-01-24
Updating pinxi in /usr/local/bin using inxi-perl branch as download source...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   125  100   125    0     0    363      0 --:--:-- --:--:-- --:--:--   363
100  626k  100  626k    0     0   770k      0 --:--:-- --:--:-- --:--:-- 3687k
-------
Downloader Data:
curl   -y 4  -L  https://github.com/smxi/inxi/raw/inxi-perl/pinxi
Result: Success: stdout data not null.
Successfully updated to inxi-perl branch version: 3.0.30
New inxi-perl branch version patch number: 05
New inxi-perl branch version release date: 2019-01-24
To run the new version, just start pinxi again.
----------------------------------------

Starting download of man page file now.
Skipping man download because branch version is being used.

Back to top
fm
Status: Curious
Joined: 01 Jan 2019
Posts: 5
Reply Quote
fm.no-ip.com/Tmp/Linux/typescript.txt is a transcript of my last attempt session. It includes ping and traceroute and --alt 4# options, from openSUSE Tumbleweed 20190121, whose rpm remains 3.0.29.
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3895
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
ping and traceroute won't show you anything, they are different protocols and tools.

Try the revised pinxi, which will now show with --dbg 1the actual curl/wget etc output.

I believe you have the same issue as that other sys admin had about 10 years or so ago, and in answer to your initial question, exactly 2 people, both running mid sized networks, have had this issue in the roughly 10 plus years I've done this stuff. I don't believe it's a coincidence that this issue is happening on a midsized network by the way, some forgotten network appliance, some directly upstream filter or misconfigured firewall, as noted, I can't diagnose such failures for you, it's not related to inxi or smxi or sgfxi, which tended to have issues more often due to the frequency of their wget requests.

I've seen individual failures, most of which I'd assume are caused by low quality LAN equipment, cluttered wifi ram / cpu in the router, etc, but not repeatable, so in a sense you have a good opportunity here to learn some fine quality networking arcana.

You can check curl versions on the affected systems and see if you find a version pattern, just to exclude it.

Again, I'm intellectually curious about what makes a midsized network decide that a certain pattern of network activity over http needs to be blocked, but it's only intellectual curiosity from my own sys admin / networking days, another small factoid to add in my knowledge base about what can make networks fail to deliver or send certain types of packet groups.

Note that you are sending an http request, so ping and traceroute are not going to do you much good except to verify that the remote host is up and running, and that the path to the host is available, but I already knew that was the case.

I don't know what apt uses for getting data, I assume it's some type of http client, unless they wrote their own, or are using Perl's build in native tools.
Back to top
fm
Status: Curious
Joined: 01 Jan 2019
Posts: 5
Reply Quote
The way I wrote about "installations" was misleading. The hundreds of installations are spread across 25+ multi multi boot PCs. No more than 6 are ever running at once. Two run 24/7. The rest are testing devices. 90% of the time there is electricity no more than 3 are running at once, plus a laptop when my sister visits, plus the A/V center STBs, which number 5 and allow no adjustment as to how their networking works except to choose between DHCP and fixed IP. Virtually everything here is on fixed IP.

Most likely every inxi I have installed was installed originally from an .rpm or .deb. In some cases where -U failed I was able to get to a newer version with an rpm.

I just tested again the TW host that lead me to start this thread, a Kaby Lake using UEFI, with same result. So I rebooted it to openSUSE 15.0, with 3.0.29 and same result, then to 15.1alpha with 3.0.29 and same result, then to Buster, which succeeded in updating from 3.0.29 to 3.0.30-00. Next came *buntu 18.04.1already on 3.0.30 but claimed success running -U anyway, but again reporting 3.0.30-00. 'which pinxi' responds null. Next was Mint Tara reporting 3.0.30-00 before and after reported -U success, and command not found to pinxi. How do I get 3.0.30-05?

I spent the next several hours repeating on a bunch of other hosts. Most of the failures were on openSUSE, but a few were on Mageia:[code]Host fi965
OS original action result
STW 3.0.30-00 -U "success" 3.0.30-00
S15.0 3.0.30-00 -U "success" 3.0.30-00
S15.1 3.0.30-00 -U "success" 3.0.30-00
Buster 3.0.30-00 -U "success" 3.0.30-00
Fedora 28 3.0.30-00 -U "success" 3.0.30-00
Fedora 29 3.0.30-00 -U "success" 3.0.30-00
Mageia 6 2.3.11 2017-05-31 -U "success" 3.0.30-00...
Checking Man page download URL...
Man file download URL failed, unable to continue: https://github.com/smxi/inxi/raw/master/inxi.1.gz

Host p5bse
S15.0 3.0.30-00 -U "success" 3.0.30-00
S15.1 3.0.30-00 -U "success" 3.0.30-00
Fedora 29 3.0.30-00 -U "success" 3.0.30-00
Buster 3.0.29-00 -U "success" 3.0.30-00

Host hpg33
STWK3 3.0.28-00 -U "success" 3.0.30-00
STWK5 3.0.30-00 -U "success" 3.0.30-00
Mageia 6 2.3.11-00 2017-05-31 -U "success" 3.0.30-00...
Checking Man page download URL...
Man file download URL failed, unable to continue: https://github.com/smxi/inxi/raw/master/inxi.1.gz
Fedora 30 3.0.30-00 -U "success" 3.0.30-00
S15.0 3.0.29-00 -U "success" 3.0.30-00
S42.3 3.0.27-00 -U "success" 3.0.30-00

Host big41
S15.0 3.0.30-00 -U "success" 3.0.30-00
S15.1 3.0.30-00 -U "success" 3.0.30-00
STWK5 3.0.30-00 -U "success" 3.0.30-00
Buster 3.0.30-00 -U "success" 3.0.30-00
Fedora 29 3.0.30-00 -U "success" 3.0.30-00
Mageia 6 2.3.11-00 2017-05-31 -U "success" 3.0.30-00...
Checking Man page download URL...
Man file download URL failed, unable to continue: https://github.com/smxi/inxi/raw/master/inxi.1.gz
*buntu 18.04.1 3.0.28-00 -U "success" 3.0.30-00
S13.1 3.0.20-00 -U "success" 3.0.30-00

Host gx745
STW 3.0.27-00 -U "success" 3.0.30-00
ST42.3 3.0.26-00 -U "success" 3.0.30-00
ST15.0 2.3.40 -U "success" 3.0.30-00...
Starting download of man page file now.
Checking Man page download URL...
Man file download URL failed, unable to continue: https://github.com/smxi/inxi/raw/master/inxi.1.gz

Host gx78b
Buster 3.0.30-00 -U "success" 3.0.30-00
S15.1 3.0.29-00 -U Starting inxi self updater.
Using curl as downloader.
Currently running inxi version number: 3.0.29
Current version patch number: 00
Current version release date: 2018-12-10
Updating inxi in /usr/bin/using main branch as download source...
Error 33: Error downloading file with curl: https://github.com/smxi/inxi/raw/master/inxi
Error: main branch
S15.0 3.0.29-00 -U Starting inxi self updater.
Using curl as downloader.
Currently running inxi version number: 3.0.29
Current version patch number: 00
Current version release date: 2018-12-10
Updating inxi in /usr/bin/using main branch as download source...
Error 33: Error downloading file with curl: https://github.com/smxi/inxi/raw/master/inxi
Error: main branch
STW 3.0.29-00 -U Starting inxi self updater.
Using curl as downloader.
Currently running inxi version number: 3.0.29
Current version patch number: 00
Current version release date: 2018-12-10
Updating inxi in /usr/bin/using main branch as download source...
Error 33: Error downloading file with curl: https://github.com/smxi/inxi/raw/master/inxi
Error: main branch

Host gb250
*buntu 18.04.1 3.0.30-00 -U "success" 3.0.30-00
Buster 3.0.29-00 -U "success" 3.0.30-00
S15.1 3.0.30-00 -U "success" 3.0.30-00
S15.0 3.0.30-00 -U "success" 3.0.30-00
STW 3.0.29-00 -U Starting inxi self updater.
Using curl as downloader.
Currently running inxi version number: 3.0.29
Current version patch number: 00
Current version release date: 2018-12-10
Updating inxi in /usr/bin/using main branch as download source...
Error 33: Error downloading file with curl: https://github.com/smxi/inxi/raw/master/inxi
Error: main branch
# inxi --dbg 1 -U
...
Downloader Data:
curl -y 4 -L -s https://github.com/smxi/inxi/raw/master/inxi
Result: Download resulted in null data!
Error 33: Error downloading file with curl: https://github.com/smxi/inxi/raw/master/inxi
Error: main branch
# zypper ref
Retrieving repository...
Download (curl) error for 'http://download.opensuse.org...
Error code: Curl error 60
Error message: SSL certificate problem: unable to get local issuer certificate
Abort, retry, ignore?...
# wget https://github.com/smxi/inxi/raw/master/inxi
--2019-01-24 22:08:16-- https://github.com/smxi/inxi/raw/master/inxi
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
ERROR: cannot verify github.com's certificate, issued by 'CN=DigiCert SHA2 Extended Validation Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US':
Unable to locally verify the issuer's authority.
To connect to github.com insecurely, use `--no-check-certificate'.
# wget --no-check-certificate https://github.com/smxi/inxi/raw/master/inxi
...
# ls -gG inxi
-rw-rw-r-- 1 640890 Jan 24 22:13 inxi
# cp -a inxi /usr/bin/
# inxi -V
inxi 3.0.30-00 (2018-12-31)[/code]
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3895
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
You're looking the wrong places, this has _nothing_ to do with the hardware, unless your nic is damaged on the machines.

Look at the curl version: curl --version

and see if you can find patterns in the version number. Ignore distro versions, only use the actual output of curl --version.

https://github.com/smxi/inxi/raw/master/inxi.1.gz

that file doesn't exist anymore, you're using legacy inxi versions that did not have the updated man downloader, you can ignore those.

again, it's not about what distro is failing, it's about the inxi version (in case of man downloads, it's trying to download a file that does not exist).

You can skip all this by just downloading pinxi and running it on the machines where inxi failed, with --dbg 1 so you can see what actually failed.

Again, this has nothing to do with inxi, it's something in these systems if your setup is how you report it.

The man is just a legacy url, that gz was removed about 1 year ago, and I assume inxi 2.3.56 etc does not know anything about that change, which is why you shouldn't run legacy inxi's in general.

Anything older than 2.3.56 inxi is silly and should be removed, just run the real inxi from github directly, don't waste your time on distro packages, they add no value except for rarely updating, and sometimes never updating in the case of stable frozen pool releases, I never use, never installed, an inxi package, no reason to.

Debian sid / testing has pretty good inxi version support, arch does ok too, some other distros update it well, others do not. No frozen pool distro, like ubuntu, opensuse, fedora, etc updates after the pool freeze, except for the rolling release variants like tumbleweed, but again, that has nothing to do with inxi.

If you read the github page, you'll note I say several times I do not support legacy inxi versions, I only support current inxi, I can't help it if distros don't update inxi, that's their policy, their choice, nothing to do with me.

Get the actual things that matter, curl version, wget versions, nothing else matters unless opensuse have a networking bug, but that's not an inxi problem either.
Back to top
Display posts from previous:   

All times are GMT - 8 Hours