Specialized Topics Continued

Networking programming
1. Essential Concepts
  1. Each system has a virtual "loopback" network adapter
  2. IP range 127.x.x.x is reserved for loopback. Default 127.0.0.1 (localhost)
  3. TCP and UDP use 216 ≈ 65000 ports
  4. Ports 1 – 1024 are reserved and on many systems need admin privileges
  5. A unique port is needed at each part of the communication
  6. Address 0.0.0.0 is used as any/all address
2. Client Networking (TCP)
  1. The client initiates the connection by opening a connection to server port
  2. The client "knows" which ip address and port to use
  3. The TCP stack takes care of providing two channels (sending and receiving)
3. Server Networking
  1. The server binds for requests on a predefined port on a predefined address (or 0.0.0.0 for all)
  2. The server then waits for a connection by calling accept to receive a new connection
  3. Accepted connections get their own local port so that multiple connections are possible
  4. Server now needs to wait for new connections as well as handle the current one
4. Sockets
  1. Abstraction for network access
  2. Origin in Unix
  3. Abstracts away from implementation and networking protocol
  4. Provides file handles that can be read or written to
  5. Sockets don't have a "file" length
  6. End of file when connection is closed
5. Client Sockets
  1. Specify address family (AF_INET, AF_INET6, AF_UNIX), and address (including port) of destination
  2. Local port can be specified, but generally isn't (OS manages this)
  3. Connect will initiate the connection
  4. Connection can then be written to or read from.
6. Server Sockets
  1. Specify address family (AF_INET, AF_INET6, AF_UNIX), and port (and address) to listen to
  2. Accept will wait for a connection and return a new open socket for the connection
  3. Connection can then be written to or read from.
7. Synchronous Networking
  1. Accept, recv, send (or read, write) are blocking (they return only when finished)
  2. Network operations can be really slow
  3. You might want to do something while waiting (like update the GUI and respond to user events) – so you need threads
8. Asynchronous Networking
  1. Give commands, the OS handles them
  2. Callbacks on completion/error
  3. Polling to see if new data is available
  4. The only way to do I/O in Javascript
  5. No synchronization problems (race conditions etc.), but harder programming model.
9. Http Basics
  1. <COMMAND> <PATH> <PROTOCOL> eg: "GET / HTTP/1.1"
  2. <HEADERS> eg: "Accept-Encoding: gzip"
10. Implementation of a server
  1. Serve "basic" HTTP/1.0
  2. Ignore most headers
  3. Return the request in text/plain
11. Server Workflow
  1. Create a new socket. Bound to 0.0.0.0:80
  2. While not shutting down
    1. accept a new connection
    2. Read the first line in the connection
    3. Read a line. If it's empty stop processing, If not, add it to the known headers
    4. If there is a content-length header. Read the content
    5. Process the request based on the path
Advanced I/O
1. What is I/O
  1. Computers only make sense if they can interact
  2. That means they need to be able to receive inputs
  3. And produce outputs
2. I/O Devices
  1. Keyboard
  2. Mouse
  3. Screen
  4. Scanner
  5. Printer
  6. Hard Disk
3. Hardware perspective
  1. CPU
    1. Devices live on buses (PCI, USB)
    2. Buses are middle-men (often layered)
    3. Devices are exposed in two ways (not exclusive)
      1. I/O ports (mostly used in Memory Mapped mode)
      2. Memory access (put video memory into cpu address space)
  2. I/O ports (on x86 architecture)
    1. Maximum 32 bits
    2. Addressed one by one
    3. In total 65,536 available (each device only gets a limited amount)
    4. Ports are unique across a computer (PnP needed to dynamically assign ports)
    5. Can be exposed as "plain" memory (memory mapped I/O)
  3. Memory based I/O
    1. Ports are made available as memory addresses (no longer a big issue with large address spaces)
    2. Device based memory is made available in main address space
    3. Creates (imperfect) illusion that everything is memory
  4. DMA (Direct Memory Access)
    1. I/O in many cases is really slow
    2. Memory mapped I/O doesn't stop slowness
    3. Copying I/O from hardware to memory is a "simple" operation
    4. Special purpose helper system (DMA engine)
    5. Hardware needs to "cooperate" / be designed for it.
  5. OS role
    1. All these technical details are primarily relevant for driver / OS writers
    2. But there are performance and design consequences that leak into programs
4. Programming I/O
  1. Kinds of I/O
    1. Disk – Arbitrated through File System layer
    2. Video – Arbitrated through windowing system
    3. Audio – Arbitrated through audio layer
    4. Network – Arbitrated through networking layer
    5. Mouse, Keyboard etc. Arbitrated through windowing system
    6. General I/O – Arbitrated through device drivers exposing specific or generic interface
  2. Common thread
    1. There can only be one program controlling a device
    2. Many programs want access
    3. The OS must arbitrate and often control
    4. Programs receive OS regulated access to smaller abstractions of the device
  3. I/O Models
    1. Synchronous – Stop the thread while performing an operation
    2. Asynchronous – Don't wait for results (but optionally be notified on completion)
  4. Access Modes
    1. Command based – Use special commands to access the I/O device
    2. Memory mapped – Get access as if it is a range in memory (where this makes sense) – A bit more complex, but faster, and reuses virtual memory techniques
  5. Java
    1. Did not use to support many advanced modes
    2. With NIO (New IO) supports many more modes
  6. I/O Buffering
    1. Unbuffered – When using small reads this can be very slow
    2. Buffered – Read in large blocks, let users use it as small
    3. Unless requesting "direct I/O" always has OS cache between program and file
  7. Access modes
    1. Read – Only read the file
    2. Write – Remove existing content and write
    3. Append – Add to the end of the file
    4. Random Access – Be able to move through the file (and reading/writing)
    5. Read+ – Read access that does not disallow writing
  8. Files kinds
    1. Binary files. Store sequences of bytes. When storing/reading values larger than bytes, byte order matters (big/small endian)
    2. Text files. Store sequences of characters. Depends on character encoding (mapping bytes in the file to unicode), and line ending
Callbacks and Lambdas
1. Functors
  1. Literally function objects
  2. Allow functionality to be provided outside object control
  3. In some cases could be replaced with inheritance
  4. uses
    1. Comparator<T>
    2. OnClickListener
    3. Runnable
  5. Difference from function
    1. Function only has address / name
    2. Functor / function object may have state
2. Function ref
  1. Address to a function
  2. Type that allows for correct calling
  3. Since 1.7 there is a MethodHandle in java that “does this”
  4. Traditional Method objects are also references
  5. In C/C++ it is a pointer (32 or 64 bits) to the address where the executable code starts
3. Poor man's objects (with virtual methods)
  1. Struct(ured) type with method pointers
  2. The pointed to functions have the object as first parameter, and operate on it
  3. The function pointers don't change (or are stored in a shared “class” object)
  4. Example
4. 2 kinds of data - java
  1. Objects – Always references, inherit from Object, all methods are virtual
  2. Primitives – Limited, fixed set of stack based numbers that are the building blocks of everything else
5. Other entities
  1. Classes/types – Have a runtime “existence” through Class objects
  2. Methods – Available through reflection
  3. You can see them, but they are not straightforward (or cheap) to use.
  4. They are second class citizens.
6. First class functions
  1. When you don't have data
  2. “like static methods”
  3. Parameters can be functions
  4. Functions can return functions
  5. Other first class "things"
    1. Primitives
      1. can be calculated
    2. Objects
      1. Can be instantiated (new Object)
  6. Creating first class functions
    1. Lambda (λ) (Yes the Greek letter)
    2. Lambda calculus is a formalisation of programming
    3. Lambda functions are functions that create functions.
7. Why lambda
  1. Lambda calculus teaches us the awesome power of “dynamic” function creation
  2. Very compact way to specify functions in-line
  3. Allows for easy use of very powerful algorithms
  4. Use new Thread(() -> { System.out.println(); });
8. Lambda usage
  1. map(iterable, function) → new iterable storing the results
  2. filter(collection, function) → shortened collection containing only some functions
  3. Event handlers
  4. Create a function that incorporates a link to an object and calls a method on it as a replacement of a functor.
Generics/Templates
1. Core programming concepts
  1. Abstraction
  2. Handling complexity
  3. Code reuse
    1. Using libraries
    2. Designing your own code
  4. Correctness
2. Design for reuse
  1. Support various uses
  2. Accept any type
  3. Keep code correct
  4. Make types and other things a part of the "parameters" to types and functions
3. Introspection
  1. Type info available at run-time
  2. Instantiate objects at run-time
  3. Create method calls at run-time
  4. Java Class objects
4. Code generation
  1. Special code
    1. Android's R.java – Link resources to java
    2. Qt's MOC (Meta Object Compiler) to support a library feature
  2. Generate code from UML (even round-tripping
  3. Compiler (generates assembler code)
5. Runtime code generation
  1. Better(reified) generics (C# style)
  2. Just in time compilation
  3. Various Java libraries with Classloader
6. Compile time code generation
  1. Special purpose tools
  2. One-off programs
  3. Framework tools
    1. Android R.java
    2. JSP processing
7. Generics
  1. Allow compiler to enforce rules on types
  2. Language specifies type dependencies in classes / functions
  3. Only work on types
  4. Generic types are highly related (but not actually the same type)
  5. One body of code
8. Generic implementation
  1. Type erasure (java) – At run-time generics have disappeared.
  2. Reification (.net) – Remember the used type, allowing for things like arrays of generics (up to having separate implementation for primitives)
9. What is an object in Java/C#
10. In place use of objects
11. Memory usage
  1. In place - Memory is part of owner (object / stack)
  2. Pointer/reference - Separately allocated memory, Owner has a pointer (size does not depend on type)
12. Templates
  1. C++ solution to code reuse (especially for containers)
  2. Unlike generics, is not a type of itself
  3. Arbitrary specialisations are allowed
  4. Tells compiler how to create function or class based on parameter type or number
13. Example Template

Next up

Specialized Topics Continued

Description

Resource summary

Media attachments

Similar

	Created by Bhavit Chhatralia almost 10 years ago