Maui Forums
[Solved] - Suspected defect using encrypted /home. - Printable Version

+- Maui Forums (https://forums.mauilinux.org)
+-- Forum: Maui Support (https://forums.mauilinux.org/forumdisplay.php?fid=74)
+--- Forum: Installation (https://forums.mauilinux.org/forumdisplay.php?fid=83)
+--- Thread: [Solved] - Suspected defect using encrypted /home. (/showthread.php?tid=23967)

Pages: 1 2

[Solved] - Suspected defect using encrypted /home. - kdemeoz - 23rd September 2016

Hello there.

You can see from my signature what my current OS' are. The quest for Lappy's new OS from Mint 17.3 KDE4 is now resolved, but i've been intensively researching, & testing in many VMs, candidates for Tower. I yearn to retain a KDE DE on Tower, & have tried Mint 18, Kubuntu 16.04, KFedora 24, KDE Neon, & Maui. Of all those, for me, Maui is easily the best... i *really* like it. I won't drag out this post by extolling all its virtues in detail, but suffice to say, from my comparative VM testing Maui is unambiguously the neatest, most featured, & most stable Plasma 5 adoption. Big kudos to the Devs.

However there are two serious issues currently preventing me taking the next step & installing it directly onto Tower's SSD, to replace Mint 17.3:

[1] I have concerns over security. Pls would you sign your ISO & Checksum files with your private PGP key, & post your public key in a top keyserver, so i can then confidently validate the ISO before i install it “for real”. I know your ISO & Checksum are currently on https site, but would prefer better security via PGP... please?

[2] I always partition my drives with separate /, /opt/, /home & swap. I strongly prefer to encrypt /home during installation, via the installer. This always worked nicely on Tower & Lappy with Mints KDE & Xfce. However it seems to be broken for Maui. Over the past 24 hours i've played a lot with this, in two separate Maui VMs:

(a) One was a "pure" Maui installation into a "virgin" VM, without encrypting /home. After the initial reboot, & continuously since, this VM has behaved magnificently. This marvellous experience was precisely what inspired me to favour Maui as my Tower's likely next OS, having already finished testing all the others.

(b) (i) The second was a VM i deliberately cloned from an existing Mint 17.3 KDE4 VM, having lots of pgms already installed & configured [& LOTS of KDE personalisations done], & varying degrees of copies of my real data. This Mint VM had encrypted /home. Following my installation of Maui into it over Mint, by formatting /, putting bootloader in /sda1, reusing /opt & /home [thus retaining its encryption], post-reboot i got to the Maui boot screen, got to the proper log-in screen, but Every Single Time [more reboots than i can count] after that, instead of the Plasma desktop appearing, the screen stays black & completely unresponsive to mouse & keyboard.

(b) (ii) In case the root-cause might have been my KDE4 config files in the reused /home conflicting with new Maui Plasma5 files, i did a reinstallation of Maui but this time formatted both / & /home [ie, blowing away my data], albeit still encrypting /home via the installer. Post-reboot, everything bad that i described above, repeated.

(b) (iii) I've tried rebooting with the ISO back in the virtual drive, accessing the installation's /etc/default/grub file & editing it to replace quiet splash with nomodeset then rebooting, but it did not help.

(b) (iv) I've reinstalled Maui in this VM, this time without encryption, & post-reboot everything was good [like the other VM].

My testing might not be forensically complete, but it does seem to indicate that Maui cannot handle an encrypted /home partition. I hope the root-cause is actually something else, as for me an encrypted partition is not negotiable; if Maui truly can't handle it, then regrettably i'd have to walk away from it.

Ideas anyone, please? Clearly i cannot proceed til these issues are solved.

..........................................................................................
Tower's SSD OS = Linux Mint x64 17.3 KDE 4.14.2... for now...
Lappy's SSD OS = Linux Mint x64  18 Xfce+Compiz [> 18/9/16]

RE: Suspected defect using encrypted /home. - leszek - 23rd September 2016

As we don't do anything special in terms of encryption it should work just like on any other Ubuntu based distribution.
I will check that particular issue. If you say encrypted /home what encryption method do you use ? LUKS or the one used by ubiquity which is ecryptfs ?

RE: Suspected defect using encrypted /home. - kdemeoz - 23rd September 2016

ecryptfs -- ie, just the standard option provided directly in your Installer, on the page where one has to enter their user name & password [from memory]. I did this step exactly as i do it on other distro installs with that same Installer [ie, simply tick the box], none of which other distros subsequently misbehave as i documented. I really look forward to your findings.

What about signing the ISOs too pls?

RE: Suspected defect using encrypted /home. - leszek - 23rd September 2016

I managed to reproduce the issue with the encrypted home option in the installer and uploaded a fixed installer to our repo.
So all you need to do is update the ubiquity package on the live system before opening up the installer.
You can achieve that for example with the terminal command
Code:
sudo apt-get update && sudo apt-get install ubiquity

Regarding ISO signing this is something we will consider for future ISO releases.
Thanks for your feedback that helped us figuring out the bug in the installer.

RE: Suspected defect using encrypted /home. - kdemeoz - 24th September 2016

Hi. Oh fabulous, your new Installer worked nicely, giving me no black screen anymore but instead the proper stable Plasma desktop. Furthermore, using this fixed Installer for a virgin installation now does allow me to encrypt /home & have a working system post-reboot, & if upgrading an existing system already containing an encrypted /home with data to Maui, both the encryption & the data were preserved. This is excellent -- MANY THANKS!

One unexpected disappointment was that as well as separate /home i also use a separate /opt partition, in which various pgms were installed in my old VM before upgrading to Maui. All these pgms vanished from /opt once Maui had replaced Mint, despite me telling the Installer to reuse /opt but NOT format it. I had thus expected the pgms to have survived, but they did not. Is that correct & normal behaviour, or might the new Installer have a problem?

Finally, with the ISO PGP signature issue, i remain apprehensive about proceeding to now install Maui in my real Tower [not just VMs as up to now]. What if your downloads page were to be hacked, & both the ISO & the hash files replaced with bad ones... how could i possibly know? Remember that earlier this year the Mint site was hacked & one ISO was compromised. Are you able to reassure me why my concern here is unnecessary?

RE: Suspected defect using encrypted /home. - leszek - 24th September 2016

Quote:Finally, with the ISO PGP signature issue, i remain apprehensive about proceeding to now install Maui in my real Tower [not just VMs as up to now]. What if your downloads page were to be hacked, & both the ISO & the hash files replaced with bad ones... how could i possibly know? Remember that earlier this year the Mint site was hacked & one ISO was compromised. Are you able to reassure me why my concern here is unnecessary?

We are aware of those risks and constantly trying to improve in that regard.

Quote:One unexpected disappointment was that as well as separate /home i also use a separate /opt partition, in which various pgms were installed in my old VM before upgrading to Maui. All these pgms vanished from /opt once Maui had replaced Mint, despite me telling the Installer to reuse /opt but NOT format it. I had thus expected the pgms to have survived, but they did not. Is that correct & normal behaviour, or might the new Installer have a problem?
As we did not change anything regarding that in the installer from the former one it might be a bug in ubiquity alltogether. As Mint also uses Ubiquity as installer do you know if it also suffers from this issue.
Usually I agree the installer should not format this partition if it isn't setup to be formatted.
We will keep an eye on it and see if it is reproducable.

RE: Suspected defect using encrypted /home. - kdemeoz - 25th September 2016

Hi. Wrt the /opt problem, i did this test, & the result was not good for Maui. Mint's installer behaved *correctly*. Here's what i did:

i took one of my several Maui VMs, loaded the Mint 17.3 KDE ISO into it, & installed Mint over the top of Maui, formatting only Root, & simply reusing the other partitions [/home being still encrypted]. Prior to this, still in Maui, i installed 5 pgms from my Deb files, which placed their files into /opt. I launched all 5 pgms to verify they were good in Maui. As 3 of these pgms were chromium-based browsers, launching them also created my user profile folders in /home/~/.config. Thus, there was now “user data” in both /home [encrypted] & /opt... Post-installation & reboot, Mint behaved correctly; though its Menu Launcher had no idea about any of my pgms, all of them were still present in /opt [exactly as should have been the case also with Maui], hence i was able to launch each pgm via its executable therein. Not that i bothered now given this was merely a test, but had this been for real, it would have been trivial to quickly recreate the relevant launchers in the Mint menu. This is how Maui also should have behaved.

Next, i wanted to reinstall Maui back over Mint in this VM, to recheck my earlier claim that the Maui Installer had incorrectly formatted /opt even though i'd told it not to. Sadly i could not proceed. Three or four attempts all gave the same error message, simply in trying to download your repaired Installer from your repo, per your previous advice. Each time this happened:

maui@maui:~$ sudo apt-get update && sudo apt-get install ubiquity
Ign:1 cdrom://Maui - Release amd64 (20160811) xenial InRelease
Hit:2 cdrom://Maui - Release amd64 (20160811) xenial Release
Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]          
Hit:5 http://ppa.launchpad.net/plasmazilla/releases/ubuntu xenial InRelease          
Get:6 http://archive.canonical.com/ubuntu xenial InRelease [11.5 kB]                
Get:7 http://archive.neon.kde.org/user xenial InRelease [8,872 B]                    
Get:8 http://packages.mauilinux.org xenial InRelease [10.4 kB]                      
Hit:9 http://ppa.launchpad.net/ubuntu-wine/ppa/ubuntu xenial InRelease              
Get:10 http://archive.canonical.com/ubuntu xenial/partner amd64 Packages [2,708 B]  
Get:11 http://archive.neon.kde.org/user xenial/main Sources [484 kB]                
Get:12 http://packages.mauilinux.org xenial/main amd64 Packages [20.9 kB]            
Get:13 http://archive.canonical.com/ubuntu xenial/partner i386 Packages [3,020 B]    
Get:14 http://packages.mauilinux.org xenial/main i386 Packages [18.4 kB]            
Get:15 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [143 kB]
Get:16 http://security.ubuntu.com/ubuntu xenial-security/main i386 Packages [139 kB]
Get:17 http://archive.neon.kde.org/user xenial/main amd64 Packages [1,275 kB]        
Get:18 http://security.ubuntu.com/ubuntu xenial-security/main amd64 DEP-11 Metadata [79.1 kB]
Get:19 http://security.ubuntu.com/ubuntu xenial-security/main DEP-11 64x64 Icons [62.9 kB]
Get:20 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [41.9 kB]
Get:21 http://security.ubuntu.com/ubuntu xenial-security/universe i386 Packages [41.9 kB]
Get:22 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 DEP-11 Metadata [2,261 B]
Get:23 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [4,105 B]
Get:24 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [1,176 B]
Get:25 http://security.ubuntu.com/ubuntu xenial-security/multiverse i386 Packages [1,340 B]
Get:26 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 DEP-11 Metadata [201 B]
Hit:27 http://archive.ubuntu.com/ubuntu xenial InRelease                            
Get:28 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]          
Get:29 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [390 kB]  
Get:30 http://archive.neon.kde.org/user xenial/main i386 Packages [137 kB]          
Get:31 http://archive.neon.kde.org/user xenial/main all Packages [137 kB]            
Get:32 http://archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages [385 kB]  
Get:33 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 DEP-11 Metadata [298 kB]
Get:34 http://archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons [184 kB]
Get:35 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [329 kB]
Get:36 http://archive.ubuntu.com/ubuntu xenial-updates/universe i386 Packages [326 kB]
Get:37 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [103 kB]
Get:38 http://archive.ubuntu.com/ubuntu xenial-updates/universe DEP-11 64x64 Icons [98.3 kB]
Get:39 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [5,492 B]
Get:40 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse i386 Packages [4,280 B]
Fetched 4,941 kB in 11s (412 kB/s)                                                  
Reading package lists... Error!
E: Read error - read (5: Input/output error)
W: You may want to run apt-get update to correct these problems
E: The package cache file is corrupted
maui@maui:~$


BTW, running apt-get update did not help.


So this is quite disappointing. Can you please advise?
=========================================================

Wrt my ongoing concern over the ISO & Checksum file integrity given no PGP signature, would you be willing to do the following for me pls? Once the current Installer errors are solved, could you pls make a new x64 ISO including the good Installer, & email me its SHA512 value? That way, whilst obviously not as good as an actual PGP sig, i would have integrity proof independent of your download page, at which point i would be comfortable to proceed installing Maui as my new OS. Pretty please?

RE: Suspected defect using encrypted /home. - leszek - 25th September 2016

Sad to hear that it didn't work for you. Regarding the problem itself I fear the is nothing we can do about at our end as it seems to fail on your end. So maybe the live system screwed up (to less free space available for the live system maybe).
Can you please try it again and see if it works. I cross my fingers for you that it will then work.

Regarding the issue with the /opt. So if I understood correctly there is no issue with the installer as it does not format the /opt partition. So your applications are still there but they don't appear in the menu ?
This would be an interesting thing to investigate as normally the ~/.local/share/applications folder will be checked for application links for the menu if reusing the /home partition.
The issue of the missing links in the menu could be explained if they install their links in /usr/share/applications.
Or weren't you able to recheck that issue ? (and it only applies to Mint)

When it comes to the ISO signing there is nothing I have to add to what I said last time.
We will evaluate that. (Its a team decision so it needs first discussing of that matter)
So please be patient about that. The same goes for a new ISO.


P.S. Just a small side note. Please use quotations or code tags for not self written text like the apt output as otherwise your suspicion level in our antispam bot will raise and it might ban you. The other upside of using code or quote tags is that it helps in terms of readability

RE: Suspected defect using encrypted /home. - kdemeoz - 25th September 2016

I must say -- i am terribly impressed with your rapid responses; this is great support -- thanks!

No, your interpretation of the info i gave you last time was not quite correct. In a nutshell, the Mint Installer behaved perfectly, the Maui one did not.

However, i now have much happier news to provide you. Somehow the troubles the Maui Installer created last time, with all those error msgs, managed to completely corrupt the Mint VM, so that eventually it wouldn't even boot. So i started over again, by reinstalling Mint clean into that VM [again ONLY formatting root, not opt & home which i simply reused], verified Mint was working after reboot & that home was still encrypted [yes] & that opt still contained my pgms from previously [yes], NEXT i loaded the Maui ISO into that Mint VM's virtual drive, launched it, downloaded the fixed Installer from your repo [no errors this time; yay], initiated the Maui Installer, formatted root & not opt & home which i simply reused again.

THIS time, after reboot, the Maui VM behaved impeccably!   Big Grin   home was still encrypted [yes] & opt still contained my pgms from previously [yes], & these pgms still correctly launched when i used their respective opt executables. Once again the menu had no items for them, but that's easily fixed manually.

So, as i said, after a LOT of hassles, this has ended up in a good place, & cleared another hurdle on the road to Maui becoming my REAL new OS. If only i could get that SHA512 email from you, i could do the upgrade tomorrow... [see, i've stopped asking about the PGP sig].   Tongue


PS -- i'm most terribly sorry about my mistake in pasting that content last time. I did not know about the rules/protocol, otherwise certainly i'd not have done it. I shall try very hard to do it right, next time.

RE: Suspected defect using encrypted /home. - leszek - 25th September 2016

As I don't have an e-mail address to you (and no apparently moderators cannot view e-mail addresses or you didn't made it public in your profile) here the sha512:

Code:
74d057fce2aa0e8a4503b7cd8ba459cf1fcd11173b3527d6e7b0372ef4eec118ce163a70bb9e1b0714c7e00f425cc402bd0e7a1edaa9055c3f3ef4bbeae86be5

Hope that helps.