Wikipedia:Reference desk/Archives/Computing/2016 March 23

Welcome to the Wikipedia Computing Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

March 23

tar x breaking symlinks?

So I had this tarfile which I wanted to extract in a certain way. The tarfile contained the usual sort of tree all under a top-level directory topdir. I needed to extract it somewhere else, so before running tar, I did

ln -s /somewhere/else topdir

My intent was that tar would notice that the top-level topdir already existed, so it wouldn't have to create it, and it would extract everything else in the tarfile down into it -- which would actually be down into /somewhere/else/.

(Don't worry about why I was doing this, or whether there might have been another way to do it. That's not the question.)

But something rather surprising happened. After running tar x, my symlink was gone, and there was a regular directory topdir in its place. Could tar have done this, and without asking or warning? I couldn't believe it, I figured I must have fumblefingered something, but I tried it several more times (rm -r'ing the tar-created topdir, and recreating my symlink) and the same thing happened every time.

Does anyone know if this is documented behavior? Does anyone know why tar insists on doing it this way? Does anyone know if there's a way of disabling this behavior?

(This was bsdtar 2.8.3 under MacOS. I'll have to try gnu tar on a Linux box, too.) —Steve Summit (talk) 14:21, 23 March 2016 (UTC)

You should read the descriptions of the --check-links, --dereference and --hard-dereference command-line options. SteveBaker (talk) 15:37, 23 March 2016 (UTC)

Thanks, but --check-links says it's for archive creation, and the other two seem to be, too, whereas I'm talking about something that happens during archive extraction. —Steve Summit (talk) 16:09, 23 March 2016 (UTC)

I tried Steve's example on a NetBSD 7.0 UNIX machine and it worked as he expected. Then I removed the symlink and created an ordinary file named "topdir", and this time when I extracted the archive it silently removed the file. That surprised me, but I guess it's consistent with the fact that I didn't use the -k option. On a Linux machine with current GNU tar I got the same result as Steve and I confirm that man tar did not document this behavior. However, in keeping with GNU practice, that is not the authoritative documentation, which is info tar or http://www.gnu.org/software/tar/manual/tar.html. And there, it mentions the option --keep-directory-symlink, which produces the old behavior where the symbolic link is followed and not clobbered. This is also documented in tar -?. --69.159.61.172 (talk) 17:11, 23 March 2016 (UTC)

Thank you! That --keep-directory-symlink option would be the perfect answer, except it's for gnu tar, which is not the tar I had the problem with. But, breaking down and studying the documentation more carefully (which I was too lazy to do before, thus this question) I find that for the tar I have, -P has the barely-documented effect of preserving the existing symlink I cared about. —Steve Summit (talk) 17:45, 23 March 2016 (UTC)

This is a good lesson to always be careful when extracting a tar file. tar tends to silently clobber files when extracting, unless you use some of the innumerable options. tar goes way back to old-school Unix, and like a lot of common Unix utilities user-friendliness isn't the first priority. The general idea is that tar files are supposed to contain exact copies of the source filesystem when extracted, symlinks, device files, and all that included. Consequently tar doesn't dereference symlinks unless you tell it to. --71.110.8.102 (talk) 03:28, 24 March 2016 (UTC)

Ironically, though, the behavior I observed was a result of tar trying to be a little more safe. What I wanted it to do (but which was not the default) was to be more "old-school". (Modulo the fact that in the really old days, symlinks didn't even exist yet, but anyway.) —Steve Summit (talk) 20:27, 24 March 2016 (UTC) updated 11:41, 25 March 2016 (UTC)

Color Palette for Maps

I asked for help on generating "consecutive" colors for a map before here[1] and got some really helpful responses. I just want to thank you guys again.

Here's the map I got[2], a map of the 50 US states colored according to their population. While California and Texas and the lowest population states looks "correct", everything looks looks off. For example, Florida looks far too light compared to California; just by looking at this map you won't get the impression that Florida has half the population of California. Likewise, Pennsylvania doesn't look like one-third the population of California. Anything between 10 million and 20 million feels much lighter than it should be.

Seems like (at least to my eyes), naively using a linearly progressing color scale will not be able to produce the right psychological results on the viewer. How do professional map makers do this? What kind of color scales do they use?

Here's the code I used to generate the 50 shades 40 shades of green (credits to Finlay McWalter):

#!/usr/bin/python3

import colorsys

steps = 40

hue = 0.328
satInitial = 1
satFinal = 0 

def frange11(x, y):
    m = steps - 1
    return ((m-i)/m*x + i/m*y for i in range(steps))

for v, sat in zip(frange11(0.5, 1.0), frange11(satInitial, satFinal)):
    r,g,b = map(lambda x: round(x*255),
                colorsys.hsv_to_rgb(hue, sat , v))
    color = '#{:02x}{:02x}{:02x}'.format(r,g,b)
    print ('<div style="width:200px; height:30px; background-color:#{:02x}{:02x}{:02x};">&nbsp;</div>'.format(r,g,b))

The output color palette definitely looks "correct", but it produces the "wrong" effect when applied to a map.

colours
The following discussion has been closed. Please do not modify it.

Johnson&Johnson&Son (talk) 14:24, 23 March 2016 (UTC)

I tried the website[3] BenRG referenced too, and it basically uses the same technique as above and gives out a similar result. Johnson&Johnson&Son (talk) 14:29, 23 March 2016 (UTC)

A few comments: 1) I don't know, that map looks decent to me, and I might have said FL had half the pop of CA. 2)Usually maps of states don't use single-hue color maps, that's usually more for things like population density, and sampled at much higher resolution. 3)There are many famous and widely used colormaps. "Jet" was developed by JPL and is used in lots of science publication, but here [4] is a contrarian view that strongly critiques Jet, and argues that cubehelix [5] is always better than Jet, though it might not always be the best. If you're stuck on single-hue, you might find more pleasing results with a log or exp mapping to saturation rather than linear. But I think for a map like this the pros use multi-hue maps and include colorbar with labeled values. SemanticMantis (talk) 14:36, 23 March 2016 (UTC)

Thanks. How do I implement an exponentially decreasing saturation in Python? I'm really new at this.

Should I go with exponential saturation + linear value or do I need exponential saturation + exponential value as well? Johnson&Johnson&Son (talk) 05:44, 24 March 2016 (UTC)

A lot of the problem of doing a simple linear interpolation between two colors is that our eyes are non-linear systems. Halving the numerical value of a color doesn't produce a color halfway between that color and black. Sure, it produces half the brightness on the screen of the monitor or the ink density on the paper - but our eyes don't perceive it as half as bright. You should (at a minimum) gamma-correct the resulting image. SteveBaker (talk) 15:32, 23 March 2016 (UTC)

Ah yes, that's the term I was forgetting: gamma correction. SemanticMantis (talk) 17:56, 23 March 2016 (UTC)

sRGB is already gamma-corrected. I don't think there's any gamma you could apply here that would make this more perceptually uniform. It would help to work in a more perceptually uniform space like Lab or Luv and then transform to sRGB, which is what the Javascript colormap generator that I suggested does. -- BenRG (talk) 19:28, 23 March 2016 (UTC)

It's not related to your question, but that frange function is awful, despite being from a highest-rated answer on StackExchange. In fact, it's giving you 41 colors instead of 40 because of roundoff error. I took the liberty of fixing it. If you actually wanted 41 colors, change steps to 41. Also, the right way to convert from 0..1 to 0..255 is round(x*255), not int(x*256). -- BenRG (talk) 19:28, 23 March 2016 (UTC)

• By the way: Why color by raw population rather than by population density? —Tamfang (talk) 08:21, 25 March 2016 (UTC)

As at least two people pointed out, this type of cartography (choropleth mapping) should never be used for absolute data like populations for the reason the Original Poster mentioned: different units' areas introduce perceived distortions. Choropleth maps should only be used for nominal data (like the party of an electoral result), ratio data (like temperatures or population densities), or ordinal data (like presidential approval ratings). Consider a map of the US showing the number of Nobel prize winners per state. Imagine that Alaska and Rhode Island have the same number. It will look bigger in Alaska, which would be wrong. If you need to depict absolute data, use proportional symbols or random dots or another method.Hayttom (talk) 15:10, 26 March 2016 (UTC)

Is there some equipament like this?

Its there some equipament like this?

People plug their (turned on) media players on the equipaments (not just one person but more than one), than then you plug this equipament on a speaker or car.

The player will play the audio coming form the first media player and after some minutes will play only the audio coming from the second, and then the audio coming from the third.....
This way people with different music tastes can share the same speaker or car speaker. By changing after X minutes, this also means the equimapent will not be biased toward long songs.201.79.54.91 (talk) 19:58, 23 March 2016 (UTC)

I don't think such a device exists, and if it did the result would be horrible to listen to. Imagine 3 people, one listens mainly to drum and bass, the other one likes hiphop and the third one likes to listen to metal. You can chose to play one of these music styles all night, which would make one of them happy. The ideal solution is to find a compromise, for example reggae, that all three can enjoy (even though it isn't their favorite type of music). But listening to partial songs of those different genres would make all three unhappy. The Quixotic Potato (talk) 21:03, 23 March 2016 (UTC)

Also, all your media players would have to be playing the whole time while only one would be actually heard, which seems terribly inefficient. At many parties I've been to we just take turns connection our phones to the bluetooth speaker and playing a song. Yes you have the problem of people not being "fair" and playing long songs or unpopular songs, but people who play long unpopular songs quickly get voted off the speaker by the majority and people who play popular songs don't get voted off, so it it ends up working mostly, as long as you have roughly similar tastes in music. I indeed like some full on death metal but I wouldn't subject a "party" to it, so I chose electronica and other more "easy listening" music that most people wont' hate. Vespine (talk) 21:38, 23 March 2016 (UTC)

"Also, all your media players would have to be playing the whole time while only one would be actually heard, which seems terribly inefficient." The point of this is that the song is not biased toward long songs, if on this system you played just one song and then played another ipod song, this system would be biased toward people that likes long songs, so the system would play a long song like an ambient song and then play a small song like a grindcore or hardcore song. Also if instead of playing one song, the system played songs for X minutes before changing to another ipod that would then be turned on just to start to play the songs of his round,this would save energy, but on long songs you would always play the first X minutes of a long song and would never listen to the rest of the song. Because of those 2 problems, I asked for this specific device, this device (change what "ipod" is playing after X seconds) is easier to be created than one that somehow detects when a song ends, or one that can actually make the ipod select a new song when its his turn.201.79.65.77 (talk) 12:53, 24 March 2016 (UTC)

I doubt anyone would make something like this - it would drive consumers up the wall to have one song cut off partway through, dropping you into the middle of another song, also partway through, in a different musical style. It makes more sense to put the 'jump' between songs - but for that to happen, this hypothetical gadget would need to know when a song ended - and it would have to be able to tell these various MP3 players to start playing at the beginning of a song. That would imply that the central gadget would have to be able to control every other kind of MP3 player remotely - and that's impossible because there are too many players already out there that don't have that capability.

The BEST way to do what you want would be to treat each MP3 player as a 'memory stick' (most of them can do that) - to plug them into an MP3 player with many USB connections and to give it a "random-play" setting that fairly picked tracks from each of the USB connections. However, I still doubt that there is such a device.

It would be fairly simple to build one using a USB hub and a tiny computer like a RaspberryPi. With a very little amount of 'shell programming' skill, I'm sure software could be built to pick the next memory device from a list, select a random track from that device, then feed it into a standard MP3 replay software package. But you'd need to be fairly desperate to have such a device to make it worthwhile. I don't think many people would want to buy one. Honestly - just have everyone wear headphones and listen to what the heck they want! SteveBaker (talk) 13:25, 24 March 2016 (UTC)

"Honestly - just have everyone wear headphones and listen to what the heck they want! " Not in parties, cars, at work...201.79.65.77 (talk) 13:58, 24 March 2016 (UTC)

Silent disco. When I am working I am usually wearing headphones. The Quixotic Potato (talk) 19:38, 28 March 2016 (UTC)