[nchntrman]

Here is an easy to follow guide that will allow you to backup your system and data areas and build Custom Firmware to restore from.

This tutorial was written to allow you to backup your current Pandigital Novel and create a custom firmware file that you can use to restore (flash) your PDN back to its current state. The idea is to mod/hack your Novel and then install all of your Base and/or Preferred Apks. Tweak your system until you have it just the way you want it to be. Then you can backup your Novel creating current system.img and userdata.img (with or without dalvik-cache) files. With these you can create your own custom firmware files. This will allow for a Computer free restore from the SDCARD to your ideal setup. When a new firmware package is released you can use the userdata.img only and have a new system release with your ideal Apks installation.

Requirements:

Windows compatible PC
Android Drivers installed (one that recognize the PD Novel and work with adb)
mkfs.yaffs2 (for arm) http://jiggawatt.org/badc0de/android/mkfs.yaffs2.arm.tar.gz
My zip file with the correct batch files (see attachment)
7zip (Win32) Download 7-Zip from SourceForge.net
md5summer (Win32) http://www.md5summer.org/md5v12011.zip
The OEM release firmware file that is installed on your Novel.

Perform all of the desired modifications and install all of your desired apks on your novel. Make your system setting adjustments and apk configurations. When you have your Novel in your desired ?restore? condition proceed with the steps below.

Part 1: Backup Current Novel Data and System

Install 7zip for Windows on your computer.
Unzip my zip file to your desired location.
Uncompress (rar) the mkfs.yaffs2 to the directory you unzipped my zip file to
copy adb.exe, AdbWinApi.dll & AdbWinUsbApi.dll to same directory
Copy the firmware file installed on your Novel into the same directory.
(I used 7_24_k firmware)
Run the FullBackup.bat (warning this will wipe your dalvik-cache after backing up)

This batch will perform the following:

copy mkfs.yaffs2 (arm) to the /cache dir of your Novel
back up your /data area to userdatabu.img
(it does this to the /cache area NOTE: 100M max avail, this means only 50M approx apks installation will work. I am working on a fix to use SDCARD or PD_Novel as dest)
pull the userdatabu.img file to the current Windows folder
delete the userdatabu.img file from the Novel
wipe the /data/dalvik-cache (freeing up a lot of space)
back up your /data area to userdatabunocache.img
(it does this to the /cache area NOTE: 100M max avail, this means only 50M approx apks installation will work. I am working on a fix to use SDCARD or PD_Novel as dest)
pull the userdatabu.img file to the current Windows folder
delete the userdatabu.img file from the Novel
back up the /system are to systembu.img
(I could not use /cache as the system size is 114M and cache has only 100M avail, this requires you to have 120M+ free in your /data area, I am working on a fix to use SDCARD or PD_Novel as dest)
pull the systembu.img to the current Windows folder
delete the systembu.img from the Novel
remove myfs.yaffs2 from the Novel

At the end of Part 1 you should have 3 files in the local directory all .img files. 2 of these will be inserted into the OEM firmware file to create your custom firmware flash file.

Part 2: Create Custom Firmware

decide which data backup you want (with or without cache)
rename the correct file to userdata.img
rename the systembu.img to system.img
run md5summer.exe and select the userdata.img and system.img files
add files and generate md5 checksum
save as cust.txt
open the cust.txt file with notepad
right click on the OEM firmware zip file, select open zip..
drag the system.img and userdata.img from the current directory into the open zip file window
select ok when prompted to replace files
in the open zip file window locate zip_check.md5, select edit file from menu
in the cust.txt file (open) select and copy the new md5 checksum for system.img
in the open zip_check.md5 select and paste the system.img checksum
in the cust.txt file (open) select and copy the new md5 checksum for userdata.img
in the open zip_check.md5 select and paste the userdata.img checksum
save and close the open zip_check.md5 file
7zip should prompt you that files have been changed, select ok to update/freshen
close 7zip (close the open OEM firmware)
this firmware file is nolonger OEM it now contains your custom restore

This resulting file can now be placed onto your SDCARD and flashed using the normal firmware flash procedure (power on + volume up)

Part 3: Creating updated firmware with newly released Firmeware updates

When a new firmware release is available from Pandigital the above process can be repeated omitting the system.img portions. You may use your userdata.img file created above, or create a new one using the backupdata.bat file I.L.O. The FullBackup.bat.

Obviously any system changes made will be lost, apks will be retained, dalvik-cache wiped userdata.img (userdatanocache.img) should be used.

thanks to roebeet for the mkfs.yaffs2 link and inspiration

This is a work in progress, the next tweak is to get access to /sdcard or /PD_Novel for image creations, removing the 100-120M limit for apk directory.

EDIT: Added backup to SDCARD capabilities.

EDIT: Added backup to PD_Novel (internal microSD) capabilities.

Added PermaLinks

This is a zip file with all the basic utilities needed. This was originally used to backup the system and data to the internal /cache area.
Everything in this zip file except the batch file is needed to perform the backups. The batch files have been replaced by the ones listed below.

Required files and batches for saving to cache: http://www.timthegeek.net/nchntr/files/BackupDataSystemCreatecustomfirmware.zip

Current Batch files:

For the regular OEM based firmwares which have the PD_Novel volume accessible use the following:
For backing up to the SDCARD: http://www.timthegeek.net/nchntr/files/BackuptoSDCARD.zip
For backing up to the PD_Novel: http://www.timthegeek.net/nchntr/files/BackuptoPDNovel.zip

For the OP based firmwares where PD_Novel is mounted as /data/app (also mmcblk0 and mmcblk1 are reversed)
OP Utilities 1: http://www.timthegeek.net/nchntr/files/PDN_OP_Stuff.zip
OP Utilities 2: http://www.timthegeek.net/nchntr/files/PDN_OP_Stuff1.zip

 

[blazingwolf]

While this does work it would be much better if we can figure out how to get clockworkmod recovery going. This would allow for complete nandroid backups so that if we hose something it is returned to the exact state of the last recovery. Plus, it gives us better alternatives if we brick the device (yes I know if it can be recovered then it is not a true brick).

 

[roebeet]

That would be a Day 2 event, I think. I assume the key here is to customize the Pandigital recovery partition into something more standardized, and no one has figured that out, yet. It would probably also kill the ability to use any future Pandigital updates, but that's not necessarily a bad thing.

 

[blazingwolf]

[quote name='roebeet;37968]It would probably also kill the ability to use any future Pandigital updates' date=' but that's no necessarily a bad thing.[/QUOTE'] Probably. Honestly though, I think we have been pretty lucky so far to not have completely killed these things with the hacking that has been going on. Saw way to much of it when the Droid was being hacked.

 

[roebeet]

Just wait until someone starts messing with the recovery and kernel partitions. Oh boy.

 

[nchntrman]

This was intended to be a little more n00b friendly. It is a part of an all-in-one application I am working on for windows. I'm working on a n00b friendly application something similar to redsn0w for the iPhone. A nice GUI interface (no batches or cmd lines required) check boxes for performing the vol-key mod, home screen + apk install, backing up, restoring, Market or Alternative, modified firmware creation, etc.. Including onscreen Instructions, maybe some neat animations and timers when appropriate.

 

[Mark Adams]

You know, there's a Terminal Emulator and a DOS Emulator for Android APK's. Some day, you're not even going to have to hook this thing to a computer to do this between WiFi tethering and backing up to an SD.

 

[nchntrman]

I figured out how to mount SDCARD in adb shell, this will now allow you to backup your 100M+ /system and 100M+ /data limited only by your SDCARD free space, using the internal memory PD_Novel is also an option now. I will try to get the updated batch file posted soon.

 

[indeana]

NCHNTRMAN -- Your procedure looks interesting and I could see how it could be very helpful in the event of a hacking catastrophe. Before I give it a whirl, I have some questions.1) When you say OEM, you are referring to any one of the FW files pulled down from the Pandigital site?2) What is the purpose of the dalvik cache -- is this where things like browsing history is stored? 2) Are there advantages to using the "restoration" file without/without the dalvik cache?3) In Part 2, are you using WinZip or 7zip?4) When the new zip file is created with the user's backup embedded in it, what name do you use to save it as (as in, do you use the same name as the original FW file--or could it be named MYFW.ZIP?)5) Will the Novel recognize MYFW.ZIP during the flash?6) Not to be pushy or anything, but how are your updates regarding the use of the SD card coming?Thanks for your help!

 

[nchntrman]

1) OEM Original Equimpent as Manufactured (Yes the firmwares from Pandigital.net)2) Many apps (apks) use the dalvik-cache, as does the system. 3) If you change any files that are cached in the dalvik-cache, you should remove them and allow them to be re-cached with the current version of the file. This occurs on startup, thus the (dis)advantage to using the nocache data backup would be a longer startup on the next boot. The advantage would be a cleaner integration with less (no) errors on the startup.4) No new .zip file is created, unless I read something incorrect. The walkthru indicates you open the original (oem) firmware with 7zip and replace the files.5) See above comment, some update files retain everything after _PD_*.zip IF i need to rename i use all lower case with special chars replaced with _ and I never have problems getting it to load (flash). I only ever have one .zip file in the SD Root also.6) I am stalled on that issue currently. I need to locate mkfs.ext2 and unionfs that are compiled for the PDN hardware. I am currently working on a scripting environment to allow for backups/firmwarebuilds and restores, on the Novel using terminal. I'm hoping someone with the apk maker can build me an apk front-end that has buttons and executes some shell scripts.

 

[cosmos]

nchntrman,Thanks for putting this together... I have really wanted to be able to do a complete backup of my system...So I attempted this tonight with mixed results.. I'm not sure what went wrong...Upon reboot some of my apps were there... some were not... It lost launcher pro. the navigation hack, root. 11 of my preinstalled apps were there... But I had around 25 installed. So a bunch are missing in action.smartbar was installed but not active. All my settings values were not what they were when I saved..My userdata img without cache was 43 MB.... system img was 129 MB.I did not attempt to calculate if I exceeded the cache size... your instructions say 50 MB... so I thought with 43 I should be OK..Any ideas ?cosmos

 

[nchntrman]

Sorry, Cosmos I'm not exactly sure what happened in your situation. How much space was free in your /Data before attempting this?The apps that were lost, were they installed in the /system/apps or the /data/apps?I believe that system settings (install un trusted sources and the like) are stored on the /system area, so the problem migh have occured there.On my Novel the system is around 114-116M, I do not have GE installed. I have updated the .bat files to use either SDCARD or PD_Novel for backups. I will post the updated files as soon as possible. I have been trying to put together a .sh file that can be run on the terminal or executed by a custom .apk which would allow us to backup on the Novel without a PC attached.

 

[cosmos]

nchntrman.....Another question...While my system was restoring it behaved strangely. The android progress bar screen came up until about 1/3 of the way thru as normal... Then the bar reset... came about 1/3 of the way again and reset again.... Got about 5 % and reset.... It did this 5 % a few more times before I canceled it. I assumed it had encountered some error.. Is this normal behavior ? Should I have let it continue doing this instead of stopping ?

 

[nvrpayretail]

nchntrman: Thanks for putting this together. Is there any way to verify the integrity of the package other than trying to flash it? I like the idea of having a point-in-time recovery file but given cosmos' experience would hate to rely on this if I don't know if it's a good, complete image. Also, I have been wanting to do some testing with the Eclair and Froyo builds the folks are working on but don't want to wipe out my PDN without a recovery solution available.

 

[nchntrman]

I added backup to the SDCARD to the original post (see link toward end). This is a much more usable alternative as there is not much (enough) space free in the default system to allow for a backup larger than a few installed apks. Nor the space for much more than a default /system size. Having GE installed causes a system size too big to backup on internal memory.This revised batch files will need to be placed in a directory that includes a working adb.exe setup. The support files in the original setup will also be required (mkfs.yaffs2.arm). There are 4 batch files included one for system only, one for data with current dalvik-cache, one for a data with a wiped cache, and one that does all 3 of the backups. Note the full backup and the backup wipe cache will remove your dalvik-cache and the novel should be rebooted after running either of those batch files. I did not add the reboot to the batch files.You must have enough free space on you SDCARD for this to work correctly. Typical system can be 112M-136M+, typical data with cache can be 40M-188M+, typical data without cache can be 20M-91M+.

 

[cosmos]

Don't let my experience deter you... I think I have a hardware error....I have a bad NAND memory block as posted elsewhere. This would cause lots of strange things... The use of SDCARD in lieu of NAND memory to create the backup should take care of this... Plus I included GE.I have been using nchntrman's experience...all in one load. It is very good.... I have customized it and will try again to create a backup using SDCARD to save the results.Will post the results tomorrow..

 

[nchntrman]

Whenever I do a firmware flash, even with an OEM file, it rarely makes it even to the 1/3 mark on the status bar before leaving the Android Recovery Screen and switching to the Blue Pandigital Screen. I have never seen it stop and start over. I have noticed with other update files I have made that the smaller the flash zip file the less the progress bar fills. A 2 meg file barely registers on the progress bar before switching to the Pandigital boot screen, and a 70M full system flash fills the progress bar somewhere between 1/4 and 1/3 of the progress bar.Hope that helps. Also if you have bad memory blocks, you might be better flashing the smallest userdata.img possible (oem nothing installed). I would assume that the Android OS filesystem can register bad blocks and mark the as such and when copying files will avoid these areas. I am not sure if the flasher program used in recovery is smart enough to do the same.

 

[nchntrman]

Added batch files to use the internal microSD /PD_Novel location for saving,storing backup images.

 

[nvrpayretail]

Thanks folks. I'll give the new SD card process a go.

 

[cosmos]

I have flashed 4-5 times since the last post... And am extremely happy with the result. The new bat file that uses SDROM avoided my bad block of memory. I was able to capture my apps from a working 7/24 flash and put them into an 8/06 flash... What I have been trying to do now for several days..Thanks nchntrman....GREAT TOOL!!!!The restore did not have my volume keys and root hacks... But I ran the fullinstall bat script and was up and running in a few minutes.Now to figure out what to do about my memory problem...