soc:2012:gnutoolchain [Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Porting newer GCC, Binutils, GDB and Upstreaming ======

Student: Ping Huang\\ 
Mentors: Arun Thomas, Ben Gras\\
git repository: [[http://git.minix3.org/?p=pkgsrc.git;a=log;h=refs/heads/minix-master|pkgsrc]]\\

===== Abstract =====

The primary goals of this project are porting GCC 4.7.0, binutils 2.22, and GDB 7.4.1 to MINIX 3, and sending the changes upstream.

===== Current Status =====

|**Done**|**Item**|**Percentage Complete**|**Comments**|
| {*} |Porting Binutils 2.22 |100%| testsuite result: 3 fails in gas, 21 fails in binutils |
| {*} |Porting dejagnu.4.4/expect-5.43 | 100% | Only root can run it successfully |
| {*} |Porting GCC 4.7.0 |100%| Only support C and C++ currently |
| {*} |Porting GDB 7.4.1 |100%| not stable |

===== Status Reports =====

==== Week 1 (May 21 - May 27) ====

Since I have built binutils-2.22 on minix3 successfully when I appled for the GSoC project, This week I tried to test it with DejaGnu framework. DejaGnu framework is dependent on expect. So this week the main work is porting expect and then DejaGnu to minix3. Work in detail are shown below.
        * Porting expect-5.43 to minix3. I have built it on minix3, as well as dejagnu-1.4.4. However, when I make check in binuitls-2.17, it failed, with the error message showing **"The system has no more ptys. Ask your system administrator to create more."**
        * Trying to build gcc44 on minix3 with pkgsrc. Although it succeed, the building process costed about 10 hours! Thanks to Arun and Ben, I found the cause is some wrong setting on VMware. After I resetted and reinstalled the system, the process of building gcc44 needed about 2 hours, and the setting is as follow:
                * CPU:  Intel vt-x/ept, 1 processor
                * MEM:  512MB
                * DISK: 20GB

==== Week 2 (May 28 - Jun 3) ====

This week I have been taking effort to make dejagnu/expect work well on minix3. According to the "no more ptys" message, firstly, I tried to increase the number of ptys by setting NR_PTYS to 64 in include/minix/config.h. Due to my ignorance to docs/UPDATE, I encountered some problems when rebuilding system. Finally, following docs/UPDATE, I have succeeded to rebuild it. However, with no change to MAKEDEV (later I learned from Ben), there are still 32 ptys in /dev. 

But the situation is not so bad, the funny thing I found is that: I can make check successfully as root, then if I switch to other users, it will fail. More oddly, if I first make check as other users, then root will also fail with "no more ptys" message. So does it in the system which NR_PTYS is 32. So I think the "no more ptys" error is not caused with the number of ptys, instead, it has something to do with the authority. I suspect this phenomenon is relataed to the system call "setpgrp" I have commmented away in the source code of expect (since minix3 doesn't support it).  

Anyway, DejaGnu works well on minix3 now, although only as root. I have tested binuitls-2.17 existing in pkgsrc and binuitls-2.22 I ported. The result is shown in the table below. 

| | binutils-2.17 |||<-3> binutils-2.22 |
| ::: | gas | ld | binutils | gas | ld | binutils |
| PASS | 104 | 27 | 4 | 257 | 55 | 13 |
| UNEXPECTED FAIL | 2 | 6 | 17 | 3 | 0 | 21 |
| EXPECTED FAIL | 0 | 1 | 0 | 0 | 5 | 0 |
| UNRESOLVED | 0 | 2 | 1 | 0 | 0 | 1 |
| UNSUPPORTED | 0 | 3 | 0 | 0 | 4 | 1 |

I glance over the UNEXPECTED FAILs in gas(3 fails) and binutils(21 fails), and don't find something that is dangerous. So Maybe binuitls-2.22 is ready for use on minix3. 

==== Week 3 (Jun 4 - Jun 10) ====

This week, first, I ruled out the possibility that the commentted "setpgrp" causes "no more ptys" error, because the "setpgrp" is never called when I preprocessed it. The cause mustbe has something to do with the authority of minix. After that I begin to move on porting gcc-4.7.0 to minix3.

Since gcc-4.7-20110716 is available in pkgsrc, I built it on minix3 successfully after add a configure argument "--enable-initfini-array" in Makefile, otherwise, the system would crash when checking whether "--enable-initfini-array" was given. Then I make check it, dejagnu ran smoothly, and the result seems not bad. I just make check with the execute directory in testsuite, the result shows below.

| | gcc | g++ | gfortran |
| PASS | 19956 | Error: Couldn't determine version of g++ | 4005 |
| UNEXPECTED FAIL | 45 | Error: Couldn't determine version of g++ | 9 |
| UNRESOLVED | 11 | Error: Couldn't determine version of g++ | 0 |

Next week, I will focus on porting the release version of gcc-4.7.0.

==== Week 4 (Jun 11 - Jun 17) ====

This week, I'm taking an effort to port gcc-4.7.0. Firstly, I enabled the language of c, c++ and fortran, when building gcc-4.7.0/libgfortran/intrinsics/chmod.c, it failed with the error message "S_IFMT undeclared". The reason is the options when compiling that file is not include -D_NETBSD_SOURSE. After that, the building process went on well. However, when I tried to bmake install, it failed again, complaining that some files about gfortran are not exsiting.

Then I decided to disable fortran, and only enable c and c++ to build a version. This time the building process went smoothly. However, the install process failed again. Then I tried to directly type "make install" in work/obj directory, instead of "bmake install" in pkgsrc/gcc47-release directory, and it succeeded. 

Next, I make check in obj/gcc, and the whole testsuite ran smoothly at the beginning, however, when it had run about 40000 testsuite, the make check process seemed suspending. until now, I don't know the reason. Then I ran some simple program, such as helloworld, and they all passed.

Now, I tried to build the system with gcc-4.7.0. First, I make some userland utils, such as ls, cat, cp, the tests passed too. and the whole commands are built successfully, however, then make install, it failed when installing MAKEDEV with error message :assertion "inited" failed: file "/usr/src/lib/libc/stdlib/_env.c", line 179, function __"allocenvvar", which I'm trying to fix__.

==== Week 5 (Jun 18 - Jun 24) ====

Continuing to the work in last week, I tried to build the system with gcc-4.7.0. I firstly tried to build minix 3.2.0, with three main errors. I posted a thread on googlegroup.

After the meeting with Arun and Ben, I switched my system to 3.2.1. Until now, I have built gcc-4.7.0 on minix 3.2.1 successfully. And I'm trying to rebuild the system. I have encountered two problems:

        * When build libraries in xz, it failed when building shared library liblzma.so.0.0, with the errors: "undefined reference to memcpy, __assert13, SHA256_INIT".
        * In order to continue the building, I commented away the libraries in external. The next problem I came across is when building in commands/ash. as building mknode, it failed with "/lib/libc.so: undefined reference to ''_localtime_r'"

I will fix the problems next week.

==== Week 6 (Jun 25 - Jul 1) ====

Thanks to Ben's effort, the two problems at last week have been solved. Now, the system can be built with gcc-4.7.0. But there are still some strange problems.

        * It cann't reboot. the system built with gcc-4.7.0 works well except that it cann't reboot and shutdown. With a reboot command, the error message is: "VM: pagefault: SIGSEGV 73140 bad addr 0x926f (dataseg); err 0x4 nopage read devman 73140 0x8048200 0x8048170 (hbp 83fc4c ?)". I have tried with gcc44, it works well, doesn't have this problem.
        * When make in drivers/memory, sometimes it will hang as in a dead loop. I digged into it and found that "mkfs.mfs image proto.gen" doesn't exit, and it actually can generate image rightly, but cann't exit.

==== Week 7 (Jul 2 - Jul 8) ====

This week I tried to fix the reboot problem. With many times' rebuilding, I found that the culprints are lib/csu and lib/libminc(both, not combination). If building the system in this way: with binuitls 2.17, compiling the two libraries with clang, and all the others with gcc47, then the system works well, without the reboot problem. Otherwise, reboot problem would occur.

I have tried to build the system with gcc 4.7 and binutils 2.22 in the way above, but use gcc44 instead of clang (it seems that ld 2.22 cannot link the object generated by clang), the system can be built but still have the reboot problem.

I have made clean packages of expect 5.43, binutils 2.22 and gcc 4.7, and have committed them to minix-master in pkgsrc.

==== Week 8, 9 (Jul 9 - Jul 22) ====

These two weeks I tried to fix the issues Ben encountered:

        * I have fixed the fixincludes problem with disable libssp and applying Arun's gcc44 patches.
        * For the depedency on gcc44, It seems that the current compiler is added to the depedencies. If I use gcc47 to compiler itself, then there's neither dependency on gcc44 nor gcc47.
        * For the "Unsupported long double format" error, I have fixed it with remove the ginclude/float.h in gcc/Makefile.in.
        * I didn't encounter neither the conflict libiberty PLIST entry problem nor iconv problem. 

I have committed the new gcc47 package to minix-master branch. With it and binutils 2.22, I can build world successful, the system works well except for the reboot problem.

==== Week 9 - 12 (Jul 23 - Aug 19) ====

This month I have finished these work:

        * Porting gdb-7.4.1 to minix. Until now, gdb-7.4.1 can be built successful on minix. I have make a simple test, it works well.
        * Wrapping the package of binutils-2.22 up based on the one in NetBSD pkgsrc repository. Unitl now, I have got a much clean package.
        * Trying to wrap the package of gcc-4.7.0 based on the one in NetBSD pkgsrc repository. Since I ported gcc47 based on minix-master branch at first, which is very different with the one in NetBSD, I think I need to port based on the NetBSD one. Now I'm trying to do this port, it will cost several days.
        * Wrapping the package of gdb-7.4.1 up based on the one in minix-master branch. Since the latest gdb package in NetBSD repository is gdb-6.2.1, while in minix-master branch it is gdb-6.8, so I based my work on gdb-6.8. Unitl now, I have got a much clean package.



===== Design  =====

Since This is a porting project, there is no a panoramic design. The project mainly consists of three parts which are listed below.

==== Porting Binutils 2.22 ====

==== Porting GCC 4.7.0 ====

==== Porting GDB 7.4.1 ====


===== Schedule =====

The following is my week-by-week timeline with milestones.

==== Pre-Coding Period (Apr 23 - May 20) ====

    * Read the OSDI book and related paper and getting a further use of MINIX 3 to familiarize with it.
    * Familiarize with the MINIX 3 community, including documentation, version control system, coding style, test system, etc.
    * Understanding the interaction between the toolchain and MINIX 3. 
    * Finish the FSF paperwork.

==== Week 1 (May 21 - May 27) ====

Port binutils 2.22 to MINIX.

This work mainly need to add Minix target description, the BFD backend for i386 Minix and the new depandencies in building, such as adding the description of bfd_elf32_i386_minix_vec, and the new elf-ifunc.o relied on.

==== Week 2 (May 28 - Jun 3) ====

Test binutils and write document of the binutils porting.

It may include porting dejagnu.

**//Milestone 1: Binutils are completed.//**

==== Week 3 (Jun 4 - Jun 10) ====

==== Week 4 (Jun 11 - Jun 17) ====

==== Week 5 (Jun 18 - Jun 24) ====

Port gcc 4.7.0 to MINIX.

This work includes adding Minix configure files and descriptions of Minix with ELF format, etc.

Key files involved are shown as following:

        * minix.h
        * minix-spec.h
        * i386-minix.h
        * t-minix
        * gcov-minix-fs-wrapper.h
        * gcov.c
        * libgcov.c
        * libjava
        * libstdg++-v3
        * other configure files

==== Week 6 (Jun 25 - Jul 1) ====

==== Week 7 (Jul 2 - Jul 8) - MIDTERM ====

Test gcc and and write document of the gcc porting.

This test will also test binutils and may be involved with some large applicaiton. So it needs two weeks.

**//Milestone 2: GCC is completed.//**

==== Week 8 (Jul 9 - Jul 15) ====

==== Week 9 (Jul 16 - Jul 22) ====

Port gdb 7.4 to MINIX.

This work mainly including adding the description of backend for Minix and the file involved with ptrace.

Key files involved are shown as following:

        * i386minix.c
        * i386minix-nat.c
        * i386minix-tdep.c
        * minix.mh
        * minix-nat.c
        * nm-minix.h
        * other configure files

==== Week 10 (Jul 23 - Jul 29) ====

Test gdb and writing document of the gdb porting.

Since gdb relys on the system call "ptrace" in Linux. To my knowledge, "ptrace" in MINIX is some different from it in Linux. So maybe I need a little more time to port and test gdb. 

**//Milestone 3: GDB is completed.//**

==== Week 11 (Jul 30 - Aug 5) ====

==== Week 12 (Aug 6 - Aug 12) ====

Scrub all of the code. Improve the documentation. And send the MINIX toolchain changes upstream.

**//Milestone 4: Send the changes upstream.//**

==== Week 13 (Aug 13 - Aug 19) - FINAL ====

Further refine the documentation of the whole toolchain.

===== Resources =====

  * http://www.gnu.org/software/binutils/
  * http://gcc.gnu.org/
  * http://www.gnu.org/software/gdb/
User Tools

Site Tools

Page Tools
