Support and replace HFS+ functionality*

Apple has a new file system, APFS, which will be replacing HFS+ (finally). This is good news, in general, but I (and others) have a vested interest in some features particular to HFS, including Finder Info and resource forks. Information is limited, but so far the news is good. Per the WWDC APFS presentation:

So HFS compatibility - if all of you have apps that run on HFS+ just fine, we intend for all of those to continue to run without any changes whatsoever on your side. So Apple File System will support and replace HFS+ functionality. And there’s an asterisk there because there’s three things that we will not support moving forward.

One of them is exchangedata. The other is searchfs. And the third is directory hardlinks for Time Machine.

But every other API and behavior will be supported just as it is on HFS+.

searchfs(2) and exchangedata(2) are system calls to “search a volume quickly” and “atomically exchange data between two files”, respectively.

So I’m optimistic Finder Info and Resource Forks will continue to work.

atan2 in k

While q/k offers all the stan­dard trigono­met­ric func­tions, it does not include atan2. A recent project (cal­cu­lat­ing head­ings between air­ports) required atan2; there­fore I needed to imple­ment it.

Wikipedia was my guide and my first attempt was a pretty straight-for­ward imple­men­ta­tion in k:

atan2:{[y;x]
    p: $[y>0;pi;y<0;0-pi;0N];
    $[
        x>0; atan[y%x];
        x<0; atan[y%x]+p;
        x=0; p%2;
        0N;
    ]
}

This works but the excite­ment faded when I real­ized $[] only works with scalars.

Re-exam­in­ing the atan2 rules, we can cre­ate this chart:

  x > 0 x < 0 x = 0
y > 0 atan[y%x] atan[y%x] + pi +pi%2
y < 0 atan[y%x] atan[y%x] - pi -pi%2
y = 0 atan[y%x] atan[y%x] + pi undefined

The most dif­fi­cult case is x = 0. For­tu­nately, k’s atan gives the cor­rect result for +in­fin­ity, -infin­ity, and null, so it can ignore x = 0 and y = 0

  x >= 0 x < 0
y >= 0 atan[y%x] atan[y%x] + pi
y < 0 atan[y%x] atan[y%x] - pi

Our only con­cern is adding (or sub­stract­ing) pi when x < 0:

atan2:{[y;x]
    fudge: pi * (x<0) * (2*y>=0)-1;
    
    fudge + atan[y%x]
}

Lemon 3/16

In March, Lemon received some per­for­mance enhance­ments in the form of left-hand-side direct opti­miza­tions. As the main­tainer (and sole user) of a c++-com­pat­i­ble lemon fork, I became inti­mately famil­iar with these changes (and even noted a cou­ple bugs).

Pre­vi­ously, you might write code like this:

%type expr {int}
expr(A) ::= expr(B) PLUS expr(C). { A = B + C; }
expr(A) ::= LPAREN expr(B) RPAREN. { A = B; }

which roughly trans­lated into this c code:

int temporaryA;
...
temporaryA = stack[0].int_value + stack[2].intValue;
stack[0].int_value = temporaryA;
....
temporaryA = stack[1].int_value;
stack[0].int_value = temporaryA;

(Ac­tual code is uglier due to unions and neg­a­tive stack off­sets.) With left-hand-side direct, the tem­po­rary vari­able can be elim­i­nated in some cir­cum­stances:

%type expr {int}
expr(A) ::= expr(A) PLUS expr(C). { A += C; }
expr(A) ::= LPAREN expr(B) RPAREN. { A = B; }

stack[0].int_value += stack[2].intValue;
....
stack[0].int_value = stack[1].int_value;

Left-hand-side direct opti­miza­tion hap­pens in 3 cir­cum­stances:

  1. If the left-most right-hand side term has the same name and type.
  2. If the left-most right-hand side term has no name
  3. There is a spe­cial `/X-over­writes-Y/ com­ment.

For lemon++, I had to dis­able the com­ment check. How­ever, in the sqlite gram­mar, most usage looks like:

transtype(A) ::= DEFERRED(X).  {A = @X; /*A-overwrites-X*/}

lemon++ tracks if the major type (@X) or minor type (X) are used and can still use left-hand-side direct opti­miza­tions in this case.

While left-hand-side direct opti­miza­tions are obvi­ously ben­e­fi­cial to lemon, they are prob­a­bly more ben­e­fi­cial to lemon++ when used with larger and more com­plex datatypes instead of inte­gers and point­ers. Pre­vi­ously code like this:

%type list {std::vector<int>}
list(A) ::= list(B) expr(C).
{ A = std::move(B); B.push_back(C.int_value); }

trans­lated to code like this:

std::vector<int> tmpA;
tmpA = std::move(stack[0].vector_int_value);
tmpA.push_back(stack[1].int_value);
yy_destructor(stack[0].vector_int_value);
yy_constructor(stack[0].vector_int_value, std::move(tmpA));

Now it can be rewrit­ten to this:

list(A) ::= list(A) expr(C).
{ A.push_back(C.int_value); }

stack[0].vector_int_value.push_back(stack[1].int_value);

MPW Make Preprocessor

It started off with this line in the MPW Make help file:

-i dirname         # additional directory to search for include files

What include files? Nor­mal make has an include direc­tive to do just that, but MPW Make didn’t sup­port it. Run­ning strings on the resource fork, I found this error:

No file name following !include

And so I ver­i­fied that !include filename worked as expected.

How­ever, I couldn’t find !include doc­u­mented any­where; not in the help file, not in the Build­ing and Man­ag­ing Pro­grams in MPW guide, and Google was help­less as well. Did !include have any secret sib­lings?

Run­ning strings again, this time look­ing just for a !, I found ref­er­ences to more key­words:

These also work as expected. !undef unde­fines a com­mand-line (-d) vari­able, allow­ing it to be re-defined later (sim­i­lar to the override direc­tive in nor­mal makes.)

These direc­tives (and using !) are sus­pi­ciously sim­i­lar to Microsoft’s NMake. I couldn’t tell you what the true ori­gin is, though.

Debugging Link

### Link: Error: Out of memory: unable to allocate resource buffer. (Error 27)  for largest module 
### Link: Errors prevented normal completion.

Well that’s not good. Especially since Link has previously been known to work and increasing the memory size didn’t help. But it’s repeatable and thus debuggable! The first step is running with --trace-tools to see if anything looks suspicious.

a065 StackSpace(093007f0)
     -> 32564
a065 StackSpace(093007f0)
     -> 32564
a065 StackSpace(093007f0)
     -> 32628
a11e NewPtr(00000000)
a11e NewPtr(00000001)
a11e NewPtr(00000001)
a002 Read(00f763ba)
     read(0008, 00f763ec, 00000400)
a002 Read(00f763ba)
     read(0008, 00f763ec, 00000400)
a9ee DECSTR68K(0000)
     NumToString(0000001b, 00fffc6a)
a11e NewPtr(00000064)
### Link: Error: Out of memory: unable to allocate resource buffer. (Error 27)  for largest module 
### Link: Errors prevented normal completion.

NewPtr with a size of 0. Quite suspicious indeed. Why would anyone do that? Let’s use the --debug debugger shell and investigate further.

a065 StackSpace(056577f0)
     -> 32628
$00F568CC   _NewPtr                                             ; A11E
Tool break: $a11e
$00F506E6   BEQ.B      $00F506EC                                ; 6704
$00F506EC   MOVE.B     $96D2(A5),D0                             ; 102D 96D2
$00F506F0   BEQ.B      $00F5074A                                ; 6758

$00F5074A   CLR.L      -(A7)                                    ; 42A7
            A7: 00fffff8

$00F5074C   MOVE.L     $971A(A5),-(A7)                          ; 2F2D 971A
            A7: 00fffff4

$00F50750   JSR        $6176(PC)                                ; 4EBA 6176
            A7: 00fffff0

$00F568C8   MOVEA.L    (A7)+,A1                                 ; 225F
            A1: 00f50754
            A7: 00fffff4

$00F568CA   MOVE.L     (A7)+,D0                                 ; 201F
            A7: 00fffff8

---------
$00F568CC   _NewPtr                                             ; A11E


] $00F568C8;l
$00F568C8   MOVEA.L    (A7)+,A1                                 ; 225F
$00F568CA   MOVE.L     (A7)+,D0                                 ; 201F
$00F568CC   _NewPtr                                             ; A11E
$00F568CE   MOVE.L     A0,(A7)                                  ; 2E88
$00F568D0   JMP        (A1) 

So this is glue code to call NewPtr with the pascal calling conventions (parameter passed on stack, return value returned on stack). $971A(A5) is the parameter (size), which is 0.

] $971A(A5);h
f70142:  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00   ................
f70152:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................

] d0
$00000000            0                                      %00000000

The NewPtr(1) calls are also based on that value – ((0 + 7) / 8) + 1 = 1

$00F5074A   CLR.L      -(A7)                                    ; 42A7
$00F5074C   MOVE.L     $971A(A5),-(A7)                          ; 2F2D 971A
$00F50750   JSR        NewPtrGlue                               ; 4EBA 6176
$00F50754   MOVE.L     (A7)+,$F084(A5)                          ; 2B5F F084
$00F50758   MOVE.L     $971A(A5),D0                             ; 202D 971A
$00F5075C   ADDQ.L     #$7,D0                                   ; 5E80
$00F5075E   DIVS.L     #$00000008,D0                            ; 4C7C 0800 00000008
$00F50766   ADDQ.L     #$1,D0                                   ; 5280
$00F50768   MOVE.L     D0,$963E(A5)                             ; 2B40 963E
$00F5076C   CLR.L      -(A7)                                    ; 42A7
$00F5076E   MOVE.L     $963E(A5),-(A7)                          ; 2F2D 963E
$00F50772   JSR        NewPtrGlue                               ; 4EBA 6154
$00F50776   MOVE.L     (A7)+,$F07E(A5)                          ; 2B5F F07E
$00F5077A   CLR.L      -(A7)                                    ; 42A7
$00F5077C   MOVE.L     $963E(A5),-(A7)                          ; 2F2D 963E
$00F50780   JSR        NewPtrGlue                               ; 4EBA 6146
$00F50784   MOVE.L     (A7)+,$96D4(A5)                          ; 2B5F 96D4
$00F50788   TST.L      $F07E(A5)                                ; 4AAD F07E
$00F5078C   SEQ.B      D0                                       ; 57C0
$00F5078E   TST.L      $96D4(A5)                                ; 4AAD 96D4
$00F50792   SEQ.B      D1                                       ; 57C1
$00F50794   OR.B       D1,D0                                    ; 8001
$00F50796   TST.L      $F084(A5)                                ; 4AAD F084
$00F5079A   SEQ.B      D1                                       ; 57C1
$00F5079C   OR.B       D1,D0                                    ; 8001
$00F5079E   BEQ.B      $00F507B0                                ; 6710
$00F507A0   MOVE.W     #$001B,-(A7)                             ; 3F3C 001B
$00F507A4   PEA        $0186(PC)                                ; 487A 0186
$00F507A8   MOVE.W     #$0004,-(A7)                             ; 3F3C 0004
$00F507AC   JSR        REPORTONEERROR                           ; 4EBA 127C

But why is it 0? I set a watchpoint and it gets set to 0…

$00F62268   MOVE.L     D0,$971A(A5)                             ; 2B40 971A

… but that’s in a routine called INITLINKER. And nobody else touches it until the NewPtr(0) call.

] pc;i
$00f6226c     16130668              %00000000111101100010001001101100
Handle $00f7c018 Pointer: $00f60000 Size: $00003854 Flags: L   R
Routine: INITLINKER+$c6

Let’s DumpCode the binary and see where else it gets set ($971A is -$68E6)

mpw DumpCode ~/mpw/Tools/Link | fgrep -A 20 -B 20 -- "-\$68E6(A5)"

000007E4: 206B 0008       ' k..'          MOVEA.L  $0008(A3),A0        
000007E8: 52A8 002E       'R...'          ADDQ.L   #$1,$002E(A0)       
000007EC: 1007            '..'            MOVE.B   D7,D0               
000007EE: 670E            'g.'            BEQ.S    *+$0010             ; 000007FE
000007F0: 42A7            'B.'            CLR.L    -(A7)               
000007F2: 2F2C 001C       '/,..'          MOVE.L   $001C(A4),-(A7)     
000007F6: 4EBA 0D90       'N...'          JSR      *+$0D92             ; 00001588
000007FA: 275F FFE4       ''_..'          MOVE.L   (A7)+,-$001C(A3)    
000007FE: 4CDF 18F0       'L...'          MOVEM.L  (A7)+,D4-D7/A3/A4   
00000802: 4E5E            'N^'            UNLK     A6                  
00000804: 4E74 0008       'Nt..'          RTD      #$0008              
00000808: 8941 4C4C       '.ALL'          PACK     D1,D4,#$4C4C        ; ALLOCENTS

0000080C: 4F43            'OC'            DC.W     $4F43               ; ????
0000080E: 454E            'EN'            DC.W     $454E               ; ????
00000810: 5453            'TS'            ADDQ.W   #$2,(A3)            
00000812: 0000 4E56       '..NV'          ORI.B    #$4E56,D0           ; 'V'
00000816: FFFC            '..'            DC.W     $FFFC               ; ????
00000818: 48E7 0018       'H...'          MOVEM.L  A3/A4,-(A7)         
0000081C: 286E 0008       '(n..'          MOVEA.L  $0008(A6),A4        
00000820: 202C 0042       ' ,.B'          MOVE.L   $0042(A4),D0        
00000824: B0AD 971A       '....'          CMP.L    -$68E6(A5),D0       
00000828: 6F06            'o.'            BLE.S    *+$0008             ; 00000830
--
00000828: 6F06            'o.'            BLE.S    *+$0008             ; 00000830
0000082A: 2B6C 0042 971A  '+l.B..'        MOVE.L   $0042(A4),-$68E6(A5) 
00000830: 296D 98D8 003A  ')m...:'        MOVE.L   -$6728(A5),$003A(A4) 
00000836: 202C 0042       ' ,.B'          MOVE.L   $0042(A4),D0        
0000083A: D1AD 98D8       '....'          ADD.L    D0,-$6728(A5)       
0000083E: 302C 0028       '0,.('          MOVE.W   $0028(A4),D0        
00000842: 41ED 9A12       'A...'          LEA      -$65EE(A5),A0       
00000846: E540            '.@'            ASL.W    #$2,D0              
00000848: 2670 0000       '&p..'          MOVEA.L  $00(A0,D0.W),A3     
0000084C: 422B 0030       'B+.0'          CLR.B    $0030(A3)           
00000850: 4CDF 1800       'L...'          MOVEM.L  (A7)+,A3/A4         
00000854: 4E5E            'N^'            UNLK     A6                  
00000856: 4E74 0004       'Nt..'          RTD      #$0004              
0000085A: 8941 4C4C       '.ALL'          PACK     D1,D4,#$4C4C        ; ALLOCDATA
0000085E: 4F43            'OC'            DC.W     $4F43               ; ????
00000860: 4441            'DA'            NEG.W    D1                  
00000862: 5441            'TA'            ADDQ.W   #$2,D1              
00000864: 0000 4E56       '..NV'          ORI.B    #$4E56,D0           ; 'V'
00000868: FFE2            '..'            DC.W     $FFE2               ; ????
0000086A: 48E7 0F18       'H...'          MOVEM.L  D4-D7/A3/A4,-(A7)   
0000086E: 422E FFE3       'B...'          CLR.B    -$001D(A6)          
00000872: 286E 0008       '(n..'          MOVEA.L  $0008(A6),A4   

So it’s set in a routine called ALLOCDATA, which is never executed. Interesting. One thing I didn’t mention is that this error came while linking a Desk Accessory, which is a driver. My previous usage was for normal applications. If we give Link a -P (for progress) flag, we learn this important fact:

Size of global data area: 0

So Link pointlessly allocates 3 pointers when there is no data. When I wrote the NewPtr code, I figured allocating a 0-length pointer was a rather silly thing to do so it just returned NULL in that case. Link was not amused. Updating NewPtr to allocate memory for 0-sized requests fixed the bug.

TCPIPDNRNameToIP Return Values

TCPIPDNRNameToIP initiates a DNS lookup. This includes calling TCPIPLogin with port 53, building the DNR message, and sending it with TCPIPSendUDP – all things that could be done with user-level code.

On success, the DNRstatus field is set to DNR_Pending.

Tool Errors

Tool Error Explanation
terrTCPIPNOTACTIVE Marinetti has not been started.
terrNOCONNECTION Marinetti is not connected to the network.
terrNODNSERVERS No DNS servers have been specified.
terrBUFFERTOOSMALL Out of memory error trying to increase the size of the DNR request queue.

TCP Errors

None.

TCPIPCancelDNR Return Values

TCPIPCancelDNR cancels a pending DNR request and sets the DNRstatus field to DNR_Cancelled (note the British spelling). The address of the dnrBuffer serves as the identifier.

Tool Errors

Tool Error Explanation
terrTCPIPNOTACTIVE Marinetti has not been started.
terrNODNRPENDING No pending DNR request for the dnrBuffer.

TCP Errors

None.

MariGNOtti 0.3

While test­ing a new ver­sion of Marinetti the other day, I noticed that MariG­NOtti no longer worked. Oops.

The prob­lem was some untested code. Specif­i­cally, I was check­ing that sin_family was AF_INET. In the real world, it’s prob­a­bly AF_UNSPEC.

So now it’s work­ing again. To cel­e­brate, I cre­ated a new release.

Skillet Buttermilk Cornbread

The corn­bread from my child­hood was based on a recipe from the back of the corn­meal can­nis­ter or per­haps a well-worn Betty Crocker Cook­book. There was more flour than corn­meal and it was cooked in a 9 x 9 pan.

This isn’t the corn­bread from my child­hood

I think this recipe orig­i­nated with The Pio­neer Woman but it was adjusted a bit since I didn’t have any milk on hand and I pre­fer a touch of sugar.

Liq­uid Ingre­di­ents

Dry Ingre­di­ents

Pre­heat oven to 400°. Place a pat of but­ter in your cast iron skil­let and then place your skil­let in the oven while it pre-heats. Com­bine liq­uid ingre­di­ents then mix in dry ingre­di­ents. Pour bat­ter into the heated skil­let (the bot­tom should begin to crust up imme­di­ate­ly) and cook for 20-25 minutes.

MFS and FAT

A while ago, I did some research into MFS, the original Macintosh File System. At the time, I was considering trying to implement a GS/OS File System Translator for it, though nothing ever came of that. MFS uses a 12-bit linked-list to store file allocation information, in a fashion similar to the MS-DOS FAT format.

This similarity is no coincidence; Over the weekend, I found this revealing passage on Andy Hertzfeld’s Folklore:

Our first file system used a simple data structure that didn’t scale well to large drives (in fact, it was suggested to us by Bill Gates in July 1981)…

Apple replaced MFS with HFS a year and a half later.

Microsoft extended FAT (FAT 8, FAT 12, FAT 16, FAT 32) as disk and file sizes grew bigger before replacing it with NTFS. FAT is still in use due to its simplicity and universal support.