Expanding extended partitions (or how to increase the space when I clone an SD to a larger one?)

I cloned an SD running raspbian from a 16GB to a 32GB one. The clone worked perfectly, however, the new card had a 13GB partition (missing out on the new space!)

The answer: fdisk.

Procedure should be to delete both partitions and then to recreate them with exact same starting sectors.

Key things to remember:
- use "-c=dos" to run legacy mode in disk
- create all logical exactly like it was before. Use expert mode to guarantee it starts in same sector.

$ sudo fdisk -l /dev/mmcblk0 -c=dos

Welcome to fdisk (util-linux 2.27.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/sda: 28.8 GiB, 30908350464 bytes, 60367872 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000ec4fb

Device     Boot   Start      End  Sectors  Size Id Type
(( make sure to do a print screen of your partitions, and the start/end sectors))

Command (m for help): d
Partition number (1,2,5, default 5): 2

Partition 2 has been deleted.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): e
Partition number (2-4, default 2):
First sector (999424-20479999, default 999424): make sure to use what it was before
Last sector, +sectors or +size{K,M,G,T,P} (1001470-20479999, default 20479999):

Created a new partition 2 of type 'Extended' and of size 27.3 GiB.

Command (m for help): n
All space for primary partitions is in use.
Adding logical partition 5
First sector (1003518-20479999, default 1003520):
Last sector, +sectors or +size{K,M,G,T,P} (1003520-20479999, default 20479999):

Created a new partition 5 of type 'Linux' and of size 9.3 GiB.

Command (m for help): x


Expert command (m for help): b
Partition number (1,2,5, default 5):
New beginning of data (1001471-20479999, default 1003520): 1001472

Expert command (m for help): r

Command (m for help): w

Now that you have room for a bigger file system, you must clean and then grow the existing file system that has been displaced by the changing boundaries of its partition. Remember to use the correct device designation.

In this example, I use mmcblk0p2 to represent Partition 2 on an internal SD card slot, but yours may differ.

 $ sudo e2fsck -f /dev/mmcblk0p2

Then resize:

 $ sudo resize2fs /dev/mmcblk0p2

Downloading large files from a server running only SSH/SCP

Another one to add to the eternal bucket of "great simple commands to keep in mind"

$ rsync --partial --progress --rsh=ssh --archive user@host:/PATH/TO/SOURCE /PATH/TO/DESTINATION

It will even show a nice progress bar ;)

Additionally, assuming you have a bunch of  folders you want to transfer, but they each have thousands of little files. . .

$ find . -type d -maxdepth 1 -mindepth 1 -exec tar cvf {}.tar {}  \;

$ find ./*.tar -exec gzip -S .gz -9 {}  \;

Then you can adjust your rsync to download all .gz files

How to generate a Dockerfile from an image?

So easy, it hurts:

$ docker pull image/I/Want/To/Inspect 
$ docker history --no-trunc image/I/Want/To/Inspect

Done

How to deal with 'no space left on device' error when using Docker on OS X

When time is money, sometimes starting from scratch is way more efficient than trying to salvage your data. Recently I got the famous 'no space left on device' when trying to build a container with python, maven and boost. I found this, but immediately imagined the amount of time/effort, and weighed out that it was not worth it.

If you are like me, then you already use Docker as a temporary system, so rebuilding images is not a big deal. In that case, I suggest you do the following:

Add the following to ~/.boot2docker/profile:

# Disk image size in MB 
DiskSize = 40000

This will increase your boot2docker VM to 40GB.

What next? Easy: you destroy everything and start from scratch.

boot2docker stop 
boot2docker destroy 
boot2docker init 
boot2docker start

On building websites: Comparing Scala/Play vs GO

I just pushed code to <https://github.com/talmai/gemawebsite> where I have the very same website coded to run on Scala 2.9.2  + Play 2.0.2, and on Go 1.3. Both code work, and generate the same output (checkout www.gemaestudio.com).

As of today, when given an option, I will choose Go over Scala anyday. The code speaks for itself. But out of curiosity, I took a look at memory and cpu requirements... (screenshots below are from the 'top' command).

VIRT stands for the virtual size of a process. In a nutshell we are talking about the sum of memory it is actually using (memory it has mapped into itself, shared libraries, and memory shared with other processes). VIRT represents how much memory the program is able to access at the present moment.

RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (This also corresponds directly to the %MEM column.) 

Scala uses 74MB of RAM (RES), but 297MB for VIRT. 

Go, as expected, uses only 5.4MB of RAM (UPDATE: to be honest, it's more like 3.8MB), but has mapped 828MB (VIRT). This I didn't expect.

Anyhow... it was fun while it lasted. The awesome part is knowing how much more RAM I got freed up on my box after this exercise. ;)

How to combine multiple worksheets into one in Excel? (using vba)

I have several excel workbooks (.xls files) and each workbook contains multiple worksheets. The number of worksheets (as well as their names) are not constant. Each worksheet is formatted in the same way (i.e: same header). Now I want to copy all the data on each worksheet and copy it into a single worksheet. 

Launch an Excel file that you want to combine other workbooks into. Click Developer > Visual Basic, a new Microsoft Visual Basic for applications window will be displayed, click Insert > Module, and input the following code into the Module. 

Oh, and before that, create a sheet called "Summary" where we will collect the info.

Sub SummarizeSheets()
    Dim ws As Worksheet
    Application.ScreenUpdating = False
    Sheets("Summary").Activate
    Path = "C:\path\to\your\files\"
    Filename = Dir(Path & "*.xls")
    Do While Filename <> ""
        Workbooks.Open Filename:=Path & Filename, ReadOnly:=True
            For Each Sheet In ActiveWorkbook.Sheets
                For Each ws In Worksheets
                    If ws.Name <> "Summary" Then
                        ws.Range("A01:AX201").Copy
                        ActiveSheet.Paste Range("A65536").End(xlUp).Offset(1, 0)
                    End If
                Next ws
            Next Sheet
        Workbooks(Filename).Close
        Filename = Dir()
    Loop
End Sub

Then click  button to run the code, and all of the worksheets (including the blank worksheets) within the workbooks have been merged into the master workbook.

References: http://www.extendoffice.com/documents/excel/456-combine-multiple-workbooks.html

bit-for-bit copy of DVD on osX (or: how to generate .iso on mac)

0) Insert DVD
1) Open up terminal
2) Find out where it was mounted (diskutil list)
3) unmount it (sudo umount /dev/disk1)
4) generate .iso (dd if=/dev/disk1 of=~/dvdImage.iso bs=2048 conv=sync,notrunc)