partitioning - In Ubuntu ISO MBR, why does a GPT partition overlap the boot partition, and make MBR uneditable?

Question and context:

In the MBR partition table of distros like for example ubuntu-16.04-desktop-amd64.iso, why does a GPT partition appear to overlap the primary boot partition? This seems to generate errors if I try, in any way, to edit the partition table.

I want to edit the MBR to add USB persistence (as described elsewhere), and must add a partition or enlarge the boot partition.

I believe this used to work, but the overlapped GPT #2 partition in this and similar distros appears to badly confuse fdisk, sfdisk, parted, gparted, and partprobe.

My machine is a MBR, and not a GPT bios.

What am I missing?

---

Here is the distro's MBR partition table (directly from the ISO file):

cat ubuntu-16.04-desktop-amd64.iso | xxd | head -32 | tail -5

gives:

00001b0: 28db 2b00 0000 0000 708e 0e0e 0000 8000  (.+.....p.......
00001c0: 0100 0058 e0fa 0000 0000 6048 2c00 00fe  ...X......`H,...
00001d0: ffff effe ffff 4411 2c00 8012 0000 0000  ......D.,.......
00001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00001f0: 0000 0000 0000 0000 0000 0000 0000 55aa  ..............U.

(To decode this see https://en.wikipedia.org/wiki/Master_boot_record.)

• The table starts at address 01be, where you see "80", for the #1 primary boot partition.




• Then notice the "ef" at address 01d2 which denotes the #2 primary partition as being type=GPT.

---

Here's a breakdown of the table by partition (little endian):

partition#1 (normal MBR):

  80        = 'boot' partition flag
  00 01 00  = starting HSC (head, sector, cylindar)
  00        =  partition type ("Empty partition entry")
  58 e0 fa  = last     HSC (head, sector, cylindar)

  0000 0000 = LBA (logical block address) of first absolute sector in the 
  6048 2c00 = number of sectors in partition

partition #2 (GPT):
  00        = non-boot partition

  fe ff ff  = starting HSC (head, sector, cylindar)
  ef        = partition type ("EFI system partition")
  fe ff ff  = last HSC

  4411 2c00 = LBA (logical block adr) of first abs sector in part.
  8012 0000 = number of sectors in partition

---

Attempts to edit partition table:

• fdisk reports that these partitions overlap. Note how the beginning of sdb2 [2927216] is inside of sdb1 [0-2955679] from

  
  
  

  sudo fdisk -l /dev/sdb
  

  
  
  

  gives:

  
  
  
  

    Disk /dev/sdb: 14.5 GiB, 15527313408 bytes, 30326784 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x40a863e7
  
    Device     Boot   Start     End Sectors  Size Id Type
    /dev/sdb1  *          0 2955679 2955680  1.4G  0 Empty
    /dev/sdb2       2927216 2931951    4736  2.3M ef EFI (FAT-12/16/32)
  
  




• From what I've read, it's ok to have a GPT partition, but not for partitions to overlap.




• When I try to dd the iso to a usb stick, the trouble begins. The stick boots Ubuntu live ok, but I can not edit the partition table without errors (that is while not booted and not mounted), no matter which partition editor I try.




• Furthermore, I can't even delete the partition table and return the USB stick to normal without the following lengthy procedure:

  
  
  

  dd if=/dev/zero of=/dev/sdb bs=[something like 512 or 2048; doesn't matter] count=[some large number like 100000^]
  

  
  
  

  ^Smaller numbers don't always seem to fix this. I suspect you need to not only wipe out the primary fs blocks, but the secondary ones also.

  
  
  
  

  and then use fdisk to make a new ms-dos partition table on the stick.

  
  
  

  Note, if I don't do the dd above to wipe before creating a new partition table, I get the following error message or similar when I try to edit the partition map (from either debian8 or ubuntu-16):

  
  
  

  Libparted Warning The driver descriptor says the physical block size
  is 2048 bytes, but Linux says it is 512 bytes.
  




• Also from ubuntu-12, I get this most enlightening message:

  
  
  

  gparted -l /dev/sdb:  libparted : 2.3 Could not stat device -l -- No
  
  such file or directory. /dev/sdb contains GPT signatures, indicating
  that it has a GPT table.  However, it does not have a valid fake msdos
  partition table, as it should.  Perhaps it was corrupted -- possibly
  by a program that doesn't understand GPT partition tables.  Or perhaps
  you deleted the GPT table, and are now using an msdos partition table.
  Is this a GPT partition table?  Both the primary and backup GPT tables
  are corrupt.  Try making a fresh table, and using Parted's rescue
  feature to recover partitions.
  

---

So how do I edit the distro's MBR?

BTW, I'm NOT asking how to fix the overlap, as this is the way the distro actually is, and from the looks of it, has been for some time, (I also looked at Ub-v16, 14 and 12), rather I'm looking to figure out how to edit it if possible.

Ideas:

• Could this be caused by too many applications of isohybrid (used to add a ms-dos MBR to a CDROM 9660 iso so it can boot on a USB)?




• Is there an other, possibly newer partition table editor to use?




• Is there a different, perhaps older, Ubuntu distro which doesn't have this issue?

Workaround

You can adjust your USB stick's partition sizes, if rather than using a USB maker tool to create the stick, you instead build your USB stick from scratch using only terminal commands. Here are detailed instructions that work if your system uses BIOS:

How to create a live Ubuntu USB drive with persistence for BIOS using only the terminal

---

Usage Tips for backing up using a USB stick: I keep two identical USB backup-OS sticks that I use to backup my main Debian dual boot W10 system. I want my system to not be running when I'm doing the backup. I also want two identical sticks, because sometimes a USB stick wears out. This allows me to easily make a new one just by a dd copy from the remaining good stick to a new stick.

They have saved me more than once when I got my system so broken that I could not fix it! And the incremental backup and restore procedures I include at this link above are pretty fast too. You should however, do a full backup from time to time, as it's possible for your current incremental to not work when you want it because a system flaw got written to it, corrupting it. I now try to do a full backup weekly.