Compare revisions

                "kind": "build",

                "kind": "build",

                "isDefault": true

                "isDefault": true

            }

            }

        },

        {

            "type": "shell",

            "label": "cargo run --example tmp",

            "command": "cargo run --example tmp",

            "problemMatcher": [

                "$rustc"

            ],

            "group": {

                "kind": "build",

                "isDefault": true

            }

        }

        }

    ]

    ]

}

}

 No newline at end of file

source = "registry+https://github.com/rust-lang/crates.io-index"

source = "registry+https://github.com/rust-lang/crates.io-index"

[[package]]

[[package]]

name = "first"

name = "crust"

version = "0.1.0"

version = "0.1.0"

dependencies = [

dependencies = [

 "nom 5.0.1 (registry+https://github.com/rust-lang/crates.io-index)",

 "nom 5.0.1 (registry+https://github.com/rust-lang/crates.io-index)",

[package]

[package]

name = "first"

name = "crust"

version = "0.1.0"

version = "0.1.0"

authors = ["Per Lindgren <per.lindgren@ltu.se>"]

authors = ["Per Lindgren <per.lindgren@ltu.se>"]

edition = "2018"

edition = "2018"

# Repo for the D7050E course 2019

The repo will be updated througout the course and includes a draft outline of the course and hints towards reaching the learning goals.

## Course Aim

Fundamental theories about computation and different models of computation. Construction of compilers. Lexical analysis, syntax analysis, and translation into abstract syntax.Regular expressions and grammars, context-free languages and grammars, lexer and parser generators. Identifier handling and symbol table organization. Type-checking, logical inference systems. Intermediate representations and transformations for different languages. Code optimization and register allocation. Machine code generation for common architectures.

In the course you will learn and develop your skills through hands on implementation work building your own complier from scratch. In this way theoretical aspects such as formal grammars, Structural Operational Semantics (SOS), and type rule formalisations becomes tangible. We will even touch upon memory safety and how guarantees can be achieved through static (compilet time) borrow checking. Compiler backend (code optimization etc.) will be discussed in context of LLVM, which you will optionally interface as a library for code generation.

## Draft outline

### W1 The big picture, parsing, semantic analysis, code generation.

Practical assigment:

- Define a minimal subset of Rust, including 

  - Function definitions

  - Commands (let, assignment, if then (else), while)

  - Expressions (includig function calls)

  - Primitive types (boolean, i32) and their literals

  - Explicit types everywhere

  - Explicit return(s)

- Begin writing a parser for expressions in Rust using `nom` (parser combinator library)

### W2 Formal languages and Structural Operational Semantics

Theory:

- Regular expressions and automata

- EBNF

- Structural Operational Semantics

Practical assignment:

- Formulate an EBNF for your language

- Continue on the parser implementation (you may use other tools)

### W3 Context Free Grammars, Push Down Automata and Type Checking

Theory:

- DFA/NFA (regular expressions)

- Push Down Automata (PDA) for Context Free Grammars (CFG)

- Typing Rules and their Derivations

Practical assignment:

- Formulate SOS rules for your language

- Finish parser

- Implement interpreter. Panic! on run-time error.

### W4 Parsing strategies, Mutability and Memory References

Theory:

- Parsing stratigies, pros and cons. L(1), LALR, Parising Expression Grammars (PEG), etc.

- Mutability and memory references

Practical assignment

- Formalize type rules for your language (optional)

- Start to implement type checker

- Extend parser/AST/interpreter to support `&` and `&mut. Panic! on run-time error.

### W5 Borrow checking

Theory:

- Linear types and memory safety

- The Rust borrow model

Practical assigmnent

- Finish type checker. (A program passing type checking should never run into panics in the interpreter due to type errors.)

- Start to implement borrow checker

### W6 LLVM

Theory:

- SSA form

- Concept of `unique`

- Code optimization techniques (performed by LLVM)

- LLVM API (a minimal subset)

Practical assignment

- Borrow checker implementation.

- Optional. Use LLVM as library for code generation.

### W7 Wrapping it up

Practical assignment

- Compiler harness (cli interface)

- Finish work on the compiler

### W8 Home Exam

You will get the home exam to work on the last weeks of the course. This may imply further theoretical exercises and experiments on your compiler.

### Examination

You will each be scheduled 30 minutes to present Your home exam to Jingsen and me, based on which Your grade will be determined. Schudule will be agreed on later using Doodle.

## Files

In this repo you find some examples using `nom` to parse expressions.

- main.rs

Simple recursive decent parsing.

- main*

Shows different approaches to introduce location information and custom error types.

- examples/aron.rs

- examples/climb.rs

Shows two approches to do precedence climbing.

---

## Your parser

- You may implement your parser using any tool of choice.

- You are NOT required to account for operator precedence in expressions, however you MUST support parantesized sub expressions. (+ for precedence towards higher grades)

- You are NOT required to account for location information, but your error messages will be better if you do. (+ for spans, towards higher grades)

- Error recovery is NOT required (+ for recovery towards higher grades)

## Your interpreter

- Your interpreter should be able to correctly execute programs according to your SOS.

- Your interpreter should panic (with an appropriote error message) when encountering an evaluation error (e.g., 1 + false)

## Your type checker

- Your type checker should reject ill-typed programs according to your typing rules.

- (+ for higher grades)

  - span information in type errors

  - multiple error reporting

  - type inference (relaxing explicit typing where possible)

## Your borrow checker

- Your borrow checker should reject borrow errors according to lexical scoping

- (+ for higher grades)

  - Non Lexical Lifetimes (likely hard)

## Your LLVM bindings (Optional)

Implement for higher grades

- Basic code generation.

- Pass `noalias` where possible allowing for better optimization (assuming your borrowchecker prevents aliasing).

- Other attributes, intrinsics, etc. that enables further LLVM optimizations.

 No newline at end of file

extern crate nom;

use crust::parse::test;

fn main() {

    test("- -1 + + 1", - -1 + 1);  // rust does not allow + as a unary op (I do ;)

    test("(-1-1)+(-1+3)", (-1 - 1) + (-1) + 3);

    // just to check that right associative works (you don't need to implement pow)

    test("2+3**2**3*5+1", 2 + 3i32.pow(2u32.pow(3)) * 5 + 1);

    test("(12*2)/3-4", (12 * 2) / 3 - 4);

    test("1*2+3", 1 * 2 + 3);

    // just to check that we get a parse error

    test("1*2+3+3*21-a12+2", 1 * 2 + 3 + 3 * 21 - 12 + 2);

}

// AST

use nom_locate::LocatedSpan;

pub type Span<'a> = LocatedSpan<&'a str>;

#[derive(Debug, Clone, Copy, PartialEq)]

pub enum Op {

    Eq,

    Neq,

    And,

    Or,

    Add,

    Sub,

    Mul,

    Div,

    Pow,

    Not,

}

type SpanOp<'a> = (Span<'a>, Op);

#[derive(Debug, Clone, PartialEq)]

pub enum Expr<'a> {

    Num(i32),

    Par(Box<SpanExpr<'a>>),

    // Identifier

    // Function application

    BinOp(Op, Box<SpanExpr<'a>>, Box<SpanExpr<'a>>),

    UnaryOp(Op, Box<SpanExpr<'a>>),

}

pub type SpanExpr<'a> = (Span<'a>, Expr<'a>);

// lib

pub mod ast;

pub mod parse;

extern crate nom;

use std::iter::Peekable;

use std::slice::Iter;

use nom::{

    branch::alt,

    bytes::complete::tag,

    character::complete::char,

    character::complete::{digit1, multispace0},

    combinator::{cut, map},

    error::ParseError,

    multi::many1,

    sequence::{delimited, preceded},

    IResult,

};

use crate::ast::{Expr, Op, Span, SpanExpr};

pub fn parse_i32(i: Span) -> IResult<Span, (Span, i32)> {

    map(digit1, |digit_str: Span| {

        (digit_str, digit_str.fragment.parse::<i32>().unwrap())

    })(i)

}

fn parse_op(i: Span) -> IResult<Span, (Span, Op)> {

    alt((

        map(tag("=="), |s| (s, Op::Eq)),

        map(tag("!="), |s| (s, Op::Neq)),

        map(tag("**"), |s| (s, Op::Pow)),

        map(tag("&&"), |s| (s, Op::And)),

        map(tag("||"), |s| (s, Op::Or)),

        map(tag("+"), |s| (s, Op::Add)),

        map(tag("-"), |s| (s, Op::Sub)),

        map(tag("*"), |s| (s, Op::Mul)),

        map(tag("/"), |s| (s, Op::Div)),

        map(tag("!"), |s| (s, Op::Not)),

    ))(i)

}

#[derive(Debug, Clone, PartialEq)]

pub enum Token<'a> {

    Num(i32),

    Par(Vec<SpanToken<'a>>),

    Op(Op),

}

type SpanToken<'a> = (Span<'a>, Token<'a>);

fn parse_terminal(i: Span) -> IResult<Span, SpanToken> {

    alt((

        map(parse_i32, |(s, v)| (s, Token::Num(v))),

        map(parse_par(parse_tokens), |(s, tokens)| {

            (s, Token::Par(tokens))

        }),

    ))(i)

}

fn parse_token(i: Span) -> IResult<Span, SpanToken> {

    preceded(

        multispace0,

        alt((map(parse_op, |(s, op)| (s, Token::Op(op))), parse_terminal)),

    )(i)

}

// I think the outer span is wrong

fn parse_tokens(i: Span) -> IResult<Span, (Span, Vec<SpanToken>)> {

    map(many1(parse_token), |tokens| (i, tokens))(i)

}

fn compute_atom<'a>(t: &mut Peekable<Iter<SpanToken<'a>>>) -> SpanExpr<'a> {

    match t.next() {

        Some((s, Token::Num(i))) => (*s, Expr::Num(*i)),

        Some((_, Token::Par(v))) => climb(&mut v.iter().peekable(), 0),

        Some((s, Token::Op(op))) => (*s, Expr::UnaryOp(*op, Box::new(climb(t, 4)))), // assume highest precedence

        _ => panic!("error in compute atom"),

    }

}

fn climb<'a>(

    t: &mut Peekable<Iter<SpanToken<'a>>>,

    min_prec: u8,

) -> SpanExpr<'a> {

    let mut result: SpanExpr = compute_atom(t);

    loop {

        match t.peek() {

            Some((s, Token::Op(op))) => {

                let (prec, ass) = get_prec(op);

                if prec < min_prec {

                    break;

                };

                let next_prec = prec

                    + match ass {

                        Ass::Left => 1,

                        _ => 0,

                    };

                t.next();

                let rhs = climb(t, next_prec);

                result = (*s, Expr::BinOp(*op, Box::new(result), Box::new(rhs)))

            }

            _ => {

                break;

            }

        }

    }

    result

}

pub fn test(s: &str, v: i32) {

    match parse_tokens(Span::new(s)) {

        Ok((Span { fragment: "", .. }, (_, t))) => {

            let mut t = t.iter().peekable();

            println!("{:?}", &t);

            let e = climb(&mut t, 0);

            println!("{:?}", &e);

            println!("eval {} {}", math_eval(&e), v);

            assert_eq!(math_eval(&e), v);

        }

        Ok((s, t)) => println!(

            "parse incomplete, \n parsed tokens \t{:?}, \n remaining \t{:?}",

            t, s

        ),

        Err(err) => println!("{:?}", err),

    }

}

// helpers

fn parse_par<'a, O, F, E>(

    inner: F,

) -> impl Fn(Span<'a>) -> IResult<Span<'a>, O, E>

where

    F: Fn(Span<'a>) -> IResult<Span<'a>, O, E>,

    E: ParseError<Span<'a>>,

{

    // delimited allows us to split up the input

    // cut allwos us to consume the input (and prevent backtracking)

    delimited(char('('), preceded(multispace0, inner), cut(char(')')))

}

fn math_eval(e: &SpanExpr) -> i32 {

    match e.clone().1 {

        Expr::Num(i) => i,

        Expr::BinOp(op, l, r) => {

            let lv = math_eval(&l);

            let rv = math_eval(&r);

            match op {

                Op::Add => lv + rv,

                Op::Sub => lv - rv,

                Op::Mul => lv * rv,

                Op::Div => lv / rv,

                Op::Pow => lv.pow(rv as u32),

                _ => unimplemented!(),

            }

        }

        Expr::UnaryOp(op, e) => {

            let e = math_eval(&e);

            match op {

                Op::Add => e,

                Op::Sub => -e,

                _ => unimplemented!(),

            }

        }

        _ => unimplemented!(),

    }

}

#[derive(Debug, Copy, Clone, PartialEq)]

enum Ass {

    Left,

    Right,

}

fn get_prec(op: &Op) -> (u8, Ass) {

    match op {

        Op::Add => (1, Ass::Left),

        Op::Sub => (1, Ass::Left),

        Op::Mul => (2, Ass::Left),

        Op::Div => (2, Ass::Left),

        Op::Pow => (3, Ass::Right),

        _ => unimplemented!(),

    }

}