Last updated 2003-04-04

1 - Differences between uClinux and Brecis funlinux.
    8k stack+brk -> 64k.
    Not using elf2flat.  (Using elf with -Wl,-q)
    Changing 64k stack to something else.
    Why pre-allocate memory for stack?
    "brk" is not used by malloc, etc.
    Stack can not grown unlimited.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
2 - Getting started with compiling things.
    After moving source from CD, build everything.
    If developing userland programs, kernel options for checking for stack
	overflow, and overwriting the first 512 bytes of memory.
    Learn how to change the kernel then build it.
    Other "make" options available.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
3 - User program modification needs.
    Things needed to modify userland programs for a non-MMU system.
    The program stack usage must be small.
    Careful with recursive programs.
    mmap() and munmap() differences.
    sys_errlist missing.  Change: sys_errlist[errno] to strerror(errno).
    No dynamic loading possible. (dlopen, etc.)
    No shared libraries.  Currently not possible.
    Some SYSVIPC available.  Kernel compiles. Usual usage appears ok.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
4 - Really nasty debugging -- documentation mainly for me.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

------------------------------------------------------------------------------
1 - Differences between uClinux and Brecis funlinux.

uClinux by default allows only 8k for stack and brk.  funlixux is 64k.
It can be changed, but each program running will have the same amount.
uClinux for non-mips allowed (supposedly) some programs to specify the
stack/brk size (elf2flat program).  This did not work for the mips.

We are now using "elf" format binaries, not "flat".

If you wish to change from 64k to something else, the file that needs to be
changed is uClinux/linux/fs/binfmt_elf.c.
	#define ELF_STACK_SIZE  (64*1024)
After changing it, go to uClinux/brecis/compilescripts and type "make linux".
[embedded systems are very worried about memory usage.  64k may be rather
large for many people.]

Why pre-allocate memory for stack?  Normal uClinux has brk and stack together.
brk space is not used unless you go out of your way to use it.  malloc() does
not use it, it does a mmap() call instead.  Since memory can not be added or
released by a program running (stack and/or growing/shrinking) -- no MMU --
and every program must run at a different address in memory (they must be
relocated at time of program load), the stack can NOT grow at run time.

------------------------------------------------------------------------------
2 - Getting started with compiling things.

You are going to want to do a few things before doing a lot with this system.
This will get you somewhat familiar with the operation of the main Makefile.

The first thing done after getting source off the CD is to build everything.
        cd uClinux/brecis/compilescripts
        make linuxall

Everything (except demo, adventure and some uC-src programs) is built.

For development, knowing when the user program has gone haywire with stack/brk
brk, and if low memory is creamed is a very good idea.
        cd uClinux/linux
        vi Makefile     (editor of your choice)
# Turn on checking for stack leaving allocated area.
CPPFLAGS += -DCHECKSTACK
# Turn on checking for first 512 bytes of staying zero (and start with zero).
CPPFLAGS += -DCHECKLOWMEMORYZERO
...  (take #'s off the two lines to enable debugging.)
        cd uClinux/brecis/compilescripts
        make redo       # redo the kernel compilation from scratch
                        # does not redo user libraries or programs.
(If you were more familiar, you would just "touch" the 3 files involved and
do a "make linux".)

To see other types of compile type "make" for a list in the compilescripts
directory.

These do less as you go down the list.
 "make everything" - does LINUXALL, demo, adventure
 "make linuxall" - to recompile all of userland, romdisk, and kernel.
 "make uclibc" - to recompile all of uclibc (libc.a), UCSRC, then ROMDISK
 "make libc" - make user library libc.a (no make clean), then UCSRC & ROMDISK 
 "make ucsrc" - to recompile user programs in uC-src, then ROMDISK.
 "make src" - to make user programs (uC-src) (no make clean), then ROMDISK
 "make linux" - to make kernel modules, linux kernel, re-use userland.
 "make romdisk" - to regenerate romdisk image (then does "make linux").
 "make image.bin" - to make linux kernel, re-use romdisk image.
  
Others of interest:
 "make CLEAN" - to clean up kernel, userland, romdisk, etc.
		NOTE: you probably wanted "make distclean".
 "make distclean" - to put everything as a clean distribution.
 `		(removes .config in uC-src.)
 "make demo" - to recompile user program "demo".
 "make adventure" - to recompile user program "adventure".

If one does a "make clean", then changes TYPE_SYSTEM to something else,
.config in uC-src remains as it was.  Suggest doing a "make distclean".

An example set of kernel and/or userland setups can be found in the
multiple_setups_and_kernel_images file found in this directory.
 
------------------------------------------------------------------------------
3 - User program modification needs.

User programs that have big arrays/structures on stack should be changed to
either "static", or malloc-ed and freed.  After changing a userland program:
        cd uClinux/brecis/compilescripts
        make src
If you have things outside the normal compile tree, then the nested
makefiles/scripts will need to be changed to cd and make the programs.
This does the "make romdisk" and "make linux" to create the image.bin.

Stupid Example:
    int banana(void) {
      char buf[8192];
      int s;

      s = read(0,buf,8192);
      ...
      return(s);
    }
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
1) malloc/free (program size does not increase much when loaded):
    int banana(void) {
      char *buf;
      int s;

      buf = (char *)malloc(8192);
      if (buf == NULL) {
	fprintf("out of memory\n");
          exit (1);
        }
        s = read(0,buf,8192);
        ...
        free(buf);
        return(s);
      }
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
2) static array (useful when not a recursive program -- size of loaded
   program increases, but no overhead for allocating/releasing memory.)
      int banana(void) {
        static char buf[8192];
        int s;

        s = read(0,buf,8192);
        ...
        return(s);
      }
------------------------------------------------------------------------------

An additional problem may be recursive algorithms.  Sorry, no recommendations.

mmap() and munmap() are much different.  mmap() can be used to allocate a
chunk of memory, but you must munmap() exactly the same memory and length.
Otherwise, you will have major problems.  This is a uClinux problem that has
been inherited.  It would be quite a bit of overhead to fix this.
    "ERROR, munmap of non-mmaped memory by process %d (%s): %p\n"
Is the message that you will see if this happens.

The array sys_errlist is missing from uclibc.  This is intentional.  Please
change sys_errlist[errno] to strerror(errno) -- as needed.

No dynamic loading possible.  dlopen and related routines do not work nor exist.

No shared libraries.  Not possible.

------------------------------------------------------------------------------
4 - Really nasty debugging -- documentation mainly for me.

a) In file uClinux/uclibc/libc/sysdeps/linux/mips/__uClibc_syscall.S comment
   in "/* #define CHECKSYSCALLREGISTERS */, and do the same to file
   uClinux/uclibc/libc/sysdeps/linux/mips/bits/syscalls.h.

   This will save every register (that is saved in kernel) and 512 bytes
   before and after the stack pointer before each system call.  Upon return
   from the system call, they are verified that they have not changed.

   Note: many programs will put arguments for the system call onto the
   stack, and the kernel will return values there. (like ioctl).
   As such, a message is printed, but the program is allowed to continue.

b) If you wish to see what arguments and environment variables are passed
   to the kernel from the boot loader, uncomment the definition 
   	/* #define MATEDSETUP y */
   in file uClinux/linux/arch/mipsnommu/kernel/setup.c

c) To verify that kmalloc-ed memory does not overlap, comment in the line
	/* #define CHECKOVERLAP */
   in file:
	uClinux/linux/mmnommu/slab.c

   If the .init section is not freed, then you can add the kernel section
   and the initial boot section to the CHECKOVERLAP section by defining
   CHECKOVERLAP in file:
	uClinux/linux/arch/mipsnommu/kernel/setup.c

   In file slab.c, you might want to enable checking if memory is used
   before allocation, and after freeing, but commenting in the line
	/* #define CONFIG_DEBUG_SLAB 1 */	/* Do all three */
   This sets released memory sections to 0x5a ('Z'), with the end of the
   section set to 0xa5.  Note: memory is always zero-ed when allocated.

   To debug kmalloc/kfree by seeing each call (address allocated/free, size,
   and address kmalloc/kfree called from), define DEBUG_KMALLOC_KFREE
   in file uClinux/linux/mmnommu/slab.c.

d) THIS FOLLOWING IS NOT A GOOD THING TO DO!  Note: you must not have the
   __init section released for memory reclaiming.

   If you are having problems with the async uart driver not finishing
   debug messages, you can change from using printk to the routine
   prom_printf.  This routine disables interrupts while putting out the
   message.

   If your assembler code uses the macro PRINT, you can change to
   prom_printf in file uClinux/linux/include/asm/asm.h.

e) If you believe that something is overwriting "low memory", and have
   defined CHECKLOWMEMORYZERO, you can call routine xxx_low_memory_zero
   passing as an argument a unique number.  At present, the number 0 is for
   checking in the mips_timer_interrupt routine, 1 is before a system call
   is executed (came from userland, saved registers, and getting ready to
   do the system call), and 2 is after the system call has been processed,
   and before returning to the user program.

f) Similar, if CHECKSTACK is defined, routine xxx_stack_check can be
   called to check if the stack pointer has gotten outside the allocated
   range.  It takes one value, the current stack pointer.  Due to the
   ability to run on different stacks, this is necessary.  See file:
   	uClinux/linux/arch/mipsnommu/brecis/generic/time.c
   for an example in "c".

g) If you have a particular location in memory that is getting overwritten,
   and have CHECKLOWMEMORYZERO defined, you can comment in the
	"/* #define      M4_811d0860 */
   in file
   	uClinux/linux/arch/mipsnommu/kernel/syscall.c
   and change the code to use the appropriate memory location.
   It will print out the value each time it changes, and save that value
   so it only gets printed once.  In addition, you can have it "stop"
   when a particular value is reached.  When this is done, you can press
   the reset button and examine memory via the boot prom.

h) To see each exception as it gets processed, comment in line
	/* #define      DEBUG_INTERRUPTS_PRINT */
   in file:
	uClinux/linux/arch/mipsnommu/kernel/entry.S
   This does not get the brecis hardware interrupts.

   To make sure the traps get to the routines to process them, comment in
	/* #define DEBUG_TRAPS_PRINT */
   in file:
	uClinux/linux/arch/mipsnommu/kernel/traps.c

   To debug unaligned access problems (or when accessing system memory
   from a user program), comment in line
	/* #define DEBUG_UNALIGNED */
   in file:
	uClinux/linux/arch/mipsnommu/kernel/unaligned.c
	
   To print out a '.' for each timer interrupt, comment in the line
	/* #define DEBUG_TIMER */
   in file:
	uClinux/linux/arch/mipsnommu/brecis/generic/time.c

   To print out a message for each hardware interrupt (timer interrupts are
   not done, they happen too often), comment in line
	/* #define DEBUG_IRQ_PRINT */
   in file:
	uClinux/linux/arch/mipsnommu/brecis/generic/mipsIRQ.S
------------------------------------------------------------------------------

End of file.

