Sunday, 15 April 2012

Itsy Forth: Implementing the Primitives

Itsy Forth is a tiny subset of the Forth programming language. So far we've looked at the Forth outer interpreter, inner interpreter and dictionary. This time we'll define the words required to complete the interpreter.

Peek and Poke

The Forth words to read and write memory are @ and !:
  • @ - ( addr -- x ) read x from addr
  • ! - ( x addr -- ) store x at addr
  • c@ - ( addr -- char ) read char from addr
( before -- after ) shows the contents of the stack before and after the word executes. Here's how @, c@ and ! are implemented. Remember we're keeping the top element of the data stack in the bx register.

        primitive '@',fetch
        mov bx,word[bx]
        jmp next

        primitive '!',store 
        pop word[bx]
        pop bx
        jmp next

        primitive 'c@',c_fetch
        mov bl,byte[bx]
        mov bh,0
        jmp next

Manipulating the Stack

  • drop - ( x -- ) remove x from the stack
  • dup - ( x -- x x ) add a copy of x to the stack
  • swap - ( x y -- y x ) exchange x and y
  • rot - ( x y z -- y z x ) rotate x, y and z

        primitive 'drop',drop
        pop bx
        jmp next

        primitive 'dup',dupe
        push bx
        jmp next

        primitive 'swap',swap
        pop ax
        push bx
        xchg ax,bx
        jmp next

        primitive 'rot',rote
        pop dx
        pop ax
        push dx
        push bx
        xchg ax,bx
        jmp next

Flow Control

if, else, then, begin and again all compile to branch or 0branch.
  • 0branch - ( x -- ) jump if x is zero
  • branch - ( -- ) unconditional jump
  • execute - ( xt -- ) call the word at xt
  • exit - ( -- ) return from the current word
The destination address for the jump is compiled in the cell straight after the branch or 0branch instruction. execute stores the return address on the return stack and exit removes it.

        primitive '0branch',zero_branch
        lodsw
        test bx,bx
        jne zerob_z
        xchg ax,si
zerob_z pop bx
        jmp next

        primitive 'branch',branch
        mov si,word[si]
        jmp next

        primitive 'execute',execute
        mov di,bx
        pop bx
        jmp word[di]

        primitive 'exit',exit
        mov si,word[bp]
        inc bp
        inc bp
        jmp next

Variables and Constants

  • tib - ( -- addr ) address of the input buffer
  • #tib - ( -- addr ) number of characters in the input buffer
  • >in - ( -- addr ) next character in input buffer
  • state - ( -- addr ) true = compiling, false = interpreting
  • dp - ( -- addr ) first free cell in the dictionary
  • base - ( -- addr ) number base
  • last - ( -- addr ) the last word to be defined

        constant 'tib',t_i_b,32768

        variable '#tib',number_t_i_b,0

        variable '>in',to_in,0

        variable 'state',state,0

        variable 'dp',dp,freemem

        variable 'base',base,10

        variable 'last',last,final

        ; execution token for constants
doconst push bx
        mov bx,word[di+2]
        jmp next

        ; execution token for variables
dovar   push bx
        lea bx,[di+2]
        jmp next

Compilation

  • , - ( x -- ) compile x to the current definition
  • c, - ( char -- ) compile char to the current definition
  • lit - ( -- ) push the value in the cell straight after lit

        primitive ',',comma
        mov ax,word[val_dp]
        xchg ax,bx
        add word[val_dp],2
        mov word[bx],ax
        pop bx
        jmp next

        primitive 'c,',c_comma
        mov ax,word[val_dp]
        xchg ax,bx
        inc word[val_dp]
        mov byte[bx],al
        pop bx
        jmp next

        primitive 'lit',lit
        push bx
        lodsw
        xchg ax,bx
        jmp next

Maths / Logic

  • + - ( x y -- z) calculate z=x+y then return z
  • = - ( x y -- flag ) return true if x=y

        primitive '+',plus
        pop ax
        add bx,ax
        jmp next

        primitive '=',equals
        pop ax
        sub bx,ax
        sub bx,1
        sbb bx,bx
        jmp next

Handling Strings

  • count - ( addr -- addr2 len ) addr contains a counted string. Return the address of the first character and the string's length
  • >number - ( double addr len -- double2 addr2 len2 ) convert string to number
addr contains a string of len characters which >number attempts to convert to a number using the current number base. >number returns the portion of the string which can't be converted, if any. If you're a Forth purist this is where Itsy starts to get ugly :-)

        primitive 'count',count
        inc bx
        push bx
        mov bl,byte[bx-1]
        mov bh,0
        jmp next

        primitive '>number',to_number
        pop di
        pop cx
        pop ax
to_numl test bx,bx
        je to_numz
        push ax
        mov al,byte[di]
        cmp al,'a'
        jc to_nums
        sub al,32
to_nums cmp al,'9'+1
        jc to_numg
        cmp al,'A'
        jc to_numh
        sub al,7
to_numg sub al,48
        mov ah,0
        cmp al,byte[val_base]
        jnc to_numh
        xchg ax,dx
        pop ax
        push dx
        xchg ax,cx
        mul word[val_base]
        xchg ax,cx
        mul word[val_base]
        add cx,dx
        pop dx
        add ax,dx
        dec bx
        inc di
        jmp to_numl
to_numz push ax
to_numh push cx
        push di
        jmp next

Terminal Input / Output

  • accept - ( addr len -- len2 ) read a string from the terminal
  • emit - ( char -- ) display char on the terminal
  • word - ( char -- addr ) parse the next word in the input buffer
accept reads a string of characters from the terminal. The string is stored at addr and can be up to len characters long. accept returns the actual length of the string.
word reads the next word from the terminal input buffer, delimited by char. The address of a counted string is returned. The string length will be 0 if the input buffer is empty.

        primitive 'accept',accept
        pop di
        xor cx,cx
acceptl call getchar
        cmp al,8
        jne acceptn
        jcxz acceptb
        call outchar
        mov al,' '
        call outchar
        mov al,8
        call outchar
        dec cx
        dec di
        jmp acceptl
acceptn cmp al,13
        je acceptz
        cmp cx,bx
        jne accepts
acceptb mov al,7
        call outchar
        jmp acceptl
accepts stosb
        inc cx
        call outchar
        jmp acceptl
acceptz jcxz acceptb
        mov al,13
        call outchar
        mov al,10
        call outchar
        mov bx,cx
        jmp next

getchar mov ah,7
        int 021h
        mov ah,0
        ret

outchar xchg ax,dx
        mov ah,2
        int 021h
        ret

        primitive 'word',word
        mov di,word[val_dp]
        push di
        mov dx,bx
        mov bx,word[val_t_i_b]
        mov cx,bx
        add bx,word[val_to_in]
        add cx,word[val_number_t_i_b]
wordf   cmp cx,bx
        je wordz
        mov al,byte[bx]
        inc bx
        cmp al,dl
        je wordf
wordc   inc di
        mov byte[di],al
        cmp cx,bx
        je wordz
        mov al,byte[bx]
        inc bx
        cmp al,dl
        jne wordc
wordz   mov byte[di+1],32
        mov ax,word[val_dp]
        xchg ax,di
        sub ax,di
        mov byte[di],al
        sub bx,word[val_t_i_b]
        mov word[val_to_in],bx
        pop bx
        jmp next

        primitive 'emit',emit
        xchg ax,bx
        call outchar
        pop bx
        jmp next

Searching the Dictionary

  • find - ( addr -- addr2 flag ) look up word in the dictionary
find looks in the Forth dictionary for the word in the counted string at addr. One of the following will be returned:
  • flag = 0, addr2 = counted string - if word not found
  • flag = 1, addr2 = call address if word is immediate
  • flag = -1, addr2 = call address if word is not immediate

        primitive 'find',find
        mov di,val_last
findl   push di
        push bx
        mov cl,byte[bx]
        mov ch,0
        inc cx
findc   mov al,byte[di+2]
        and al,07Fh
        cmp al,byte[bx]
        je findm
        pop bx
        pop di
        mov di,word[di]
        test di,di
        jne findl
findnf  push bx
        xor bx,bx
        jmp next
findm   inc di
        inc bx
        loop findc
        pop bx
        pop di
        mov bx,1
        inc di
        inc di
        mov al,byte[di]
        test al,080h
        jne findi
        neg bx
findi   and ax,31
        add di,ax
 inc di
        push di
        jmp next

Initialisation

  • abort - ( -- ) initialise Itsy then jump to interpret
abort initialises the stacks and a few variables before running the outer interpreter. When Itsy first runs it jumps to abort to set up the system.

        primitive 'abort',abort
        xor ax,ax
        mov word[val_state],ax
        mov word[val_to_in],ax
        mov word[val_number_t_i_b],ax
        xchg ax,bp
        mov sp,-256
        mov si,xt_interpret+2
        jmp next

Up and Running?

Itsy is now around 900 bytes and it's time to give the interpreter a quick test run:
Itsy Forth
Everything seems to be working fine. Next we'll define the compiler so we can continue building Itsy from the Itsy prompt :-)

19 comments:

  1. Great to see a Forth resource like this one that I saw on Reddit.

    ReplyDelete
    Replies
    1. Thanks Mentifix, and thanks to everyone who made Itsy Forth popular on Reddit :-)

      Delete
  2. Well done John. I bet it was a great feeling when the interpreter output "Itsy". Keep up the good work.

    ReplyDelete
    Replies
    1. Thanks Lawrence. This was actually the second attempt. The first test output "ystI" :-)

      Delete
  3. Could you please provide an example in Visual Basic .NET. It is not a very good practice to use assembly language in high level software.

    ReplyDelete
    Replies
    1. Would GW-Basic be okay?

      Delete
    2. Tsk, tsk. This is *retro* programming. Nothing later than IBM BASICA, surely?

      Delete
  4. Feel free to post the complete source code for Itsy Forth :)

    ReplyDelete
    Replies
    1. Hopefully it won't be long before the complete source is ready. I just need to tie up some loose ends. :-)

      Delete
  5. Thanks for these posts - fascinating!

    How did you get this up and running when (unless I've missed it) you don't yet have definitions of the control strucutres (IF, AGAIN) used by the outer interpreter? Did you hand-compile the 0BRANCH calls in the outer interpreter?

    ReplyDelete
    Replies
    1. Thank you, I'm pleased you enjoyed them :-)

      I compiled the outer interpreter by hand. Also : and ; will be hand compiled. After that I'm hoping everything will be defined from the Itsy prompt.

      Delete
  6. Thanks for these posts - I had forgotten how much fun Forth and asm are :)

    Can't count be implemented with extant primitives?

    : count ( addr -- addr2 len ) dup 1 + swap c@ ;

    ReplyDelete
    Replies
    1. Hi Eventi, it's all down to size. The assembly code is 4 bytes shorter than the compiled Forth :-)

      Delete
  7. It's been a very long time since I've done any programming. This does look like a good language to learn.

    ReplyDelete
  8. hi................
    fantastic post .I really impressed to see your way of explaining codes .can you tell me what to do to learn the assembly language.

    ReplyDelete
  9. Hi John,

    I believe there's a minor bug in Itsy Forth's >number routine. If it completes successfully, it returns four words on the top of the stack, just as you describe in your article: the converted double-precision number, the leftover string pointer, and a 0 that's left after counting through the string. However, if it fails due to a non-numeric string, it does not push both words of the double back onto the stack; it only pushes the high-order word, followed by the string pointer and the non-zero
    counter value. Three words rather than four. This isn't a problem for interpret, which almost immediately calls abort and clears the stack out anyway, but for somebody else who might try to use >number independently, this could be the source of some serious hair-ripping frustration. It looks to me like it would be easy to fix this simply by changing the "jc to_numh" line to read "jc to_numz".

    Mike

    ReplyDelete
    Replies
    1. Okay, it's time for an OOPS! moment. Seems I made my previous comment a bit too hastily. I noticed after the fact that there was another push to the stack BEFORE the jump in question, and it does indeed put the low word of the double number back on the stack as it should if >number aborts. The modification I suggest to the code is not necessary; in fact, making that patch would be a bug in and of itself. The code returns the correct number of items on the stack as it is, regardless of whether it encounters an illegal character or not. I cobbled together some simple numeric output routines and confirmed this.

      However, just to save you from repeating another mistaken assumption I made, let me warn you that the number at the top of the stack after >number is executed (the length counter) is not a reliable indicator of whether >number has successfully converted a numeric input string. Under some circumstances (such as if the very last character in the string is non-numeric), it can have a value of zero even if >number detects an error and aborts. Looking at the code now, I don't think John intended that number to be used for error-checking as I'd originally thought. It's just a counter, not a fault flag. It would be nice to have one available, though. I may see if I can modify the code so that it can be used as such.

      My apologies for any confusion my earlier post may have caused.

      Delete
    2. Hi Mike,

      Thanks for the bug report, I'm not sure how it slipped through. It can be fixed simply by moving dec bx further down. I've fixed the code above.

      John

      Delete