Strange NSNumber conversion behaviour in Swift

I have been doing some numerical work lately and see some strange behaviour when using NSNumber in Swift especially in the conversions to/from the int types.

First some expected behaviour in finding the integer ranges:

print("Int8 range: \(Int8.min) ... \(Int8.max)")

print("UInt8 range: \(UInt8.min) ... \(UInt8.max)")

print("Int16 range: \(Int16.min) ... \(Int16.max)")

print("UInt16 range: \(UInt16.min) ... \(UInt16.max)")

print("Int32 range: \(Int32.min) ... \(Int32.max)")

print("UInt32 range: \(UInt32.min) ... \(UInt32.max)")

print("Int64 range: \(Int64.min) ... \(Int64.max)")

print("UInt64 range: \(UInt64.min) ... \(UInt64.max)")

This prints:

Int8 range: -128 ... 127

UInt8 range: 0 ... 255

Int16 range: -32768 ... 32767

UInt16 range: 0 ... 65535

Int32 range: -2147483648 ... 2147483647

UInt32 range: 0 ... 4294967295

Int64 range: -9223372036854775808 ... 9223372036854775807

UInt64 range: 0 ... 18446744073709551615

As I said, that was to be expected. But pay attention to the Int64 and UInt64

Now some expected behaviour for NSNumber:

var n = NSNumber(value: 0x1020304050607080)

print("Int8:   \(n.int8Value)")

print("UInt8:  \(n.uint8Value)")

print("Int16:  \(n.int16Value)")

print("UInt16: \(n.uint16Value)")

print("Int32:  \(n.int32Value)")

print("UInt32: \(n.uint32Value)")

print("Int64:  \(n.int64Value)")

print("UInt64: \(n.uint64Value)")

Prints:

Int8:   -128

UInt8:  128

Int16:  28800

UInt16: 28800

Int32:  1348497536

UInt32: 1348497536

Int64:  1161981756646125696

UInt64: 1161981756646125696

Interesting, here we see that the integers don't clip (or clamp) to max, but are in effect simple bit mapped conversions. For example the -128 for the Int8 is clearly caused by the ....80 at the end of the literal.

We can make this even clearer by:

var n = NSNumber(value: 0x1020304080608080)

...

Output:

Int8:   -128

UInt8:  128

Int16:  -32640

UInt16: 32896

Int32:  -2141159296

UInt32: 2153808000

Int64:  1161981757451436160

UInt64: 1161981757451436160

So far so good.
So what would we expect if the we change 0x1020.. to 0x8020..
I don't know about you, but I would expect the Int64 to go negative and the UInt64 to remain positive.

However:

var n = NSNumber(value: 0x8020304080608080)

Generates the error:

Playground execution failed: error: MyPlayground8.playground:8:25: error: integer literal '9232432289699364992' overflows when stored into 'Int'

So let's fix that by:

var n = NSNumber(value: UInt64(0x8020304080608080))

No such luck, the same error occurs.
Next step:

var k: UInt64 = 0x8020304080608080

var n = NSNumber(value: k)

...

Int8:   -128

UInt8:  128

Int16:  -32640

UInt16: 32896

Int32:  -2141159296

UInt32: 2153808000

Int64:  -9214311784010186624

UInt64: 9232432289699364992

So far, the only strange thing is that I cannot initialize NSNumber directly from an integerLiteral. Not nice, but hey, no big deal.

Now, what would happen when NSNumber is initialized from a float or double?

Try that:

var n = NSNumber(value: 10.0)

...

Int8:   10

UInt8:  10

Int16:  10

UInt16: 10

Int32:  10

UInt32: 10

Int64:  10

UInt64: 10

Ok, next try:

var n = NSNumber(value: -10.0)

...

Int8:   -10

UInt8:  246

Int16:  -10

UInt16: 65526

Int32:  -10

UInt32: 4294967286

Int64:  -10

UInt64: 18446744073709551606

Aha, apparently the float is first converted to an Int and then a simple bit map conversion is made. This true even for the UInt64 (!)

How about rounding?

var n = NSNumber(value: 10.8)

...

Int8:   10

UInt8:  10

Int16:  10

UInt16: 10

Int32:  10

UInt32: 10

Int64:  10

UInt64: 10

Nope, no rounding here (expected behaviour)

var n = NSNumber(value: -10.8)

...

Int8:   -10

UInt8:  246

Int16:  -10

UInt16: 65526

Int32:  -10

UInt32: 4294967286

Int64:  -10

UInt64: 18446744073709551605

This may seem strange, but I am content to call that expected behaviour. As long as we keep in mind that rounding down is always in the direction of 0. Even for negative numbers.

Now, lets get wild:

var n = NSNumber(value: 1e10)

...

Int8:   0

UInt8:  0

Int16:  -7168

UInt16: 58368

Int32:  1410065408

UInt32: 1410065408

Int64:  10000000000

UInt64: 10000000000

Ok, not all that wild, still as expected. Lets get wilder still. But keep in mind that 100.000.000 is a special number, it translates to 0x5F5E100, which is to say that all multiples of that number will also result in 0xXXXXXX00. Thus simply multiplying 100.000.000 by *10 and again, and again will eventually create 0 values for all Int's.
Weird things start to happen at 1e19:

var n = NSNumber(value: 1e19)

...

Int8:   0

UInt8:  0

Int16:  0

UInt16: 0

Int32:  0

UInt32: 0

Int64:  -9223372036854775808

UInt64: 10000000000000000000

The 0 values are explainable (see above). The UInt64 value is correct, but the Int64 is odd, it seems clipped to me. (Compare it to the ranges above)
And this behaviour continues as we use ever greater numbers. And with much bigger numbers the clipping behaviour also occurs in the UInt64 (at 1e39):

var n = NSNumber(value: 1e39)

...

Int8:   0

UInt8:  0

Int16:  0

UInt16: 0

Int32:  0

UInt32: 0

Int64:  -9223372036854775808

UInt64: 18446744073709551615

Now the behaviour is odd: For an NSNumber with a very large positive value the smaller int's are zero, the Int64 is clipped against its negative (minimum) limit and the UInt64 is clipped to its maximum limit.

Well, now you know...

Happy coding...

Did this help?, then please help out a small independent.
If you decide that you want to make a small donation, you can do so by clicking this
link: a cup of coffee ($2) or use the popup on the right hand side for different amounts.
Payments will be processed by PayPal, receiver will be sales at balancingrock dot nl
Bitcoins will be gladly accepted at: 1GacSREBxPy1yskLMc9de2nofNv2SNdwqH

We don't get the world we wish for... we get the world we pay for.

Optionals and weak references | Swift gotcha

Thanks to the swift user list I became aware of the following gotcha that can lead to runtime errors of the "unexpectedly nil" kind.

The problem is as follows:

First optionals, when using optionals the following is totally safe:

var a: Int?

if a != nil { a! = a! + 1 }

Before using 'a' it is tested on non-nil and then used by force-unwrap.

No problem there.

But consider the following with weak references:

class C {

    var a: Int = 0

}

var c: C?

let b = {

    [weak c] () -> Int in

    if c != nil { return c!.a + 1 }

    return 0

}

This may bomb at runtime. It will often work quite alright, but sometimes it may crash your app. If your luck runs like Murphy's, the first crash will come right after you released the app... And you will be quite unable to reproduce it.

The problem here is that in between the test for non-nil and the usage of 'c' itself, 'c' may have become nil.

This problems only occurs in multithreading applications, which is to say in (almost?) all of them. It happens when one thread nils the referenced variable and another thread is using that variable.

While there are situations where this will work correctly, the guiding principle should be:

Never force unwrap a weak reference.

First create a strong reference to it, then access it through the strong reference.

Like this:

class C {

    var a: Int = 0

}

var c: C?

let b = {

    [weak c] () -> Int in

    guard let c = c else { return 0 }

    return c.a + 1

}

Quick tip: if [weak self] is used, then use back-ticks on self, like this:

    [weak self] () -> Int in

    guard let `self` = self else { return 0 }

Happy coding...

Did this help?, then please help out a small independent.
If you decide that you want to make a small donation, you can do so by clicking this
link: a cup of coffee ($2) or use the popup on the right hand side for different amounts.
Payments will be processed by PayPal, receiver will be sales at balancingrock dot nl
Bitcoins will be gladly accepted at: 1GacSREBxPy1yskLMc9de2nofNv2SNdwqH

We don't get the world we wish for... we get the world we pay for.

Swiftfire | Building websites in Swift

As you could guess from a lack of postings lately, I have been quite busy. The main effort has gone into a new kind of (web)server that would allow me to write websites in Swift.

Finally I have now gotten it to the point where this is now actually possible.

Swiftfire is the name of the project and it is hosted on github.

How does it work?

You can use function calls in HTML or CSS files that Swiftfire will replace with the result of that function call.

An example:

<p>This page has been accessed .nofPageHits() times<p>

will become:

<p>This page has been accessed 12744 times<p>

Assuming of course that the function "nofPageHits" has been implemented and registered with Swiftfire. (It has, because it is a build-in function)

Swiftfire is distributed as source code, allowing anybody to write their own functions, and thus allows anyone to develop entire websites in Swift.

Swiftfire uses only three hooks to allow anybody to extend its functionality:

1) Application Setup
2) Service registration
3) Function registration

The difference between Services and Functions is that functions work at the file level, and services work at the (HTTP) request level. Together they allow fully customization of the website.

As proud as I am on the current status of the project, I should point out that it is still in beta stadium. The foundation is in place, but now it needs fleshing out.

Still, if you want to create websites in Swift, head over to the swiftfire website for more information.

In case you were wondering, Swiftfire also supports HTTPS and multiple domains.

Happy coding...

Did this help?, then please help out a small independent.
If you decide that you want to make a small donation, you can do so by clicking this
link: a cup of coffee ($2) or use the popup on the right hand side for different amounts.
Payments will be processed by PayPal, receiver will be sales at balancingrock dot nl
Bitcoins will be gladly accepted at: 1GacSREBxPy1yskLMc9de2nofNv2SNdwqH

We don't get the world we wish for... we get the world we pay for.

Removing force unwrap from Swift code

Swift is a nice language, and it has introduced new concepts to many people. The optionals is probably the No 1 most visible aspect of that.

I like optionals, but sometimes... they bite... usually at runtime after the software has been released.

Crashes are the result.

I am just guessing, but it would not surprise me if most runtime crashes are caused by force-unwrapped optionals.

This may well be the reason that force-unwrap is frowned upon.

Friends don't let friends force unwrap...

With that in mind, lets purge our code from the force unwrap. In this post a few methods for doing so.

The most obvious method to use is the 'guard' method:

func check(parameter: Int?) -> Int? {

    guard let parameter = parameter else { return nil }

    ...

}

After this invocation of guard, the parameter is nicely unwrapped.

Next up is the ?? operator:

var count: Int?

var unwrappedCount = count ?? 0

By using the ?? operator we can specify a default value to use if the optional is nil

This operator is also useful in many other situation, for example the loop:

var count: Int?

for index in 0 ..< count ?? 0 { ... }

But sometimes we have optional array's,... no problem:

var counters: [Int]?

for count in counters ?? [] { ... }

The next pattern is a matter of taste, some people like it, others don't.

For this pattern first look at the check function that was defined above. It returns an optional. Obviously it is the intention to return nil if the check fails.

When using that function it will be necessary to either use 'if let' or 'guard' or the '??' operator in series with an extra assignment. Each of these ways leads to its own (taste) problems.

The 'if let' means that we will need to put the rest of the code in brackets and have to indent it.

var a: Int?

var pb: Int

if let b = check(parameter: a) {

    pb = b

} else {

    print("error")

    pb = 16 // safe value

}

That looks horrible to me.

The 'guard' in the middle of a code block is also frowned upon, but more problematic is that it is not allowed to continue after the 'else' of the guard is executed:

var a: Int?

guard var pb = check(parameter: a) {

    print("error")

    pb = 16 // safe value

    return

}

Not a real solution either.

The '??' operator has other problems:

var a: Int?

var pb = check(parameter: a) ?? 16

We lose the ability to handle the error. Unless we start using an if statement for that, but then we can do without the ??.

Which lead me to start using a generic type for function results that may generate an error:

enum FunctionResult<T> {

    case error(String)

    case success(T)

}

func check(parameter: Int?) -> FunctionResult<Int> {

    guard let parameter = parameter else { return .error("Parameter is nil") }

    if parameter < 0 { return .error("Parameter to small") }

    else { return .success(parameter) }

}

Of course we then need to use a switch:

var pb: Int

switch check(parameter: a) {

case .error(let m): print(m); pb = 16 // safe value

case .success(let t): pb = t

}

Which imo already beats the previous attempts. But we can do better... with thanks to O.B. on the swift-user mailing list, it is possible to write a generic for that:

func assign<T>(_ closure: @autoclosure() -> FunctionResult<T>, onError: (String) -> T) -> T {

    switch closure() {

    case .error(let message): return onError(message)

    case .success(let t): return t

    }

}

True, that is becoming quite an amount of code, but we only need to write it once to cover a lot of usage, and look what happens to our example:

var a: Int?

var pb = assign( check(parameter: a), onError: { print($0); return 16 } )

We don't lose any flexibility and it avoids force-unwrapping. But it may obfuscate the assignment a little.

Happy coding...

Did this help?, then please help out a small independent.
If you decide that you want to make a small donation, you can do so by clicking this
link: a cup of coffee ($2) or use the popup on the right hand side for different amounts.
Payments will be processed by PayPal, receiver will be sales at balancingrock dot nl
Bitcoins will be gladly accepted at: 1GacSREBxPy1yskLMc9de2nofNv2SNdwqH

We don't get the world we wish for... we get the world we pay for.

Swift and OpenSSL, part 5: Server side

With parts 1 - 4, I would expect people to be able to work their way through. However for the server side there is one more thing to discuss if it is necessary to host multiple domain on a single server on a single port.

But first things first, setting up a server using openSSL is almost simpler than a client.

The only things we need to do is calling SSL_accept after the accept POSIC accept call returns a socket for a new connection.

    let result = tipAccept(onSocket: acceptSocket, timeout: timeout, addressHandler: addressHandler)

    

    switch result {

    case .closed: return .closed

    case let .error(msg): return .error(message: msg)

    case .timeout: return .timeout

    case let .accepted(receiveSocket, clientIp):

(PS: I am using code excerpts from SecureSockets which uses SwifterSockets as a wrapper for the POSIX calls)

Of course we also need to set up an SSL_CTX (i.e. context) for the server to use. Lets do that first:

    guard let context = SSL_CTX_new(TLS_server_method()) else { ... }

    

    SSL_CTX_set_options(context, (UInt(SSL_OP_NO_SSLv2) + UInt(SSL_OP_NO_SSLv3) + UInt(SSL_OP_ALL)))

    

    if SSL_CTX_use_certificate_file(context, encodedFileC.path, encodedFileC.encoding) != 1 {

        return .error(message: ...)

    }

        

    if SSL_CTX_use_PrivateKey_file(context, encodedFileK.path, encodedFileK.encoding) != 1 {

        return .error(message: "...")

    }

    

The above code snippet is raked together from different files, it is just intended to show the calls involved in setting up a context.

If the server only wants to serve clients with a specific certificate, then we also have to setup those certificate paths:
        

    var isDirectory: ObjCBool = false

    if FileManager.default.fileExists(atPath: clientCertPath, isDirectory: &isDirectory) {

                        

        if isDirectory.boolValue {

                

            if SSL_CTX_load_verify_locations(context, nil, clientCertPath) != 1 {

                    

                return .error(message: "...")

            }

                

        } else {

                

            if SSL_CTX_load_verify_locations(context, clientCertPath, nil) != 1 {

                    

                return .error(message: "...")

            }

        }

    }

    SSL_CTX_set_verify(context, SSL_VERIFY_PEER + SSL_VERIFY_FAIL_IF_NO_PEER_CERT, nil)

Now the context is ready for use and we can start accepting SSL connections. Keep in mind that the SSL handshake is initiated by the client.

        SSL_ACCEPT_LOOP: while true {

            

            

            // ===================================

            // Try to establish the SSL connection

            // ===================================

                        

            let result = ssl.accept()

            switch result {

                

                

            // On success, return the new SSL structure

            case .completed:

                

                return .accepted(ssl: ssl, socket: receiveSocket, clientIp: clientIp)

                

                

            // Exit if the connection closed (i.e. there is no secure connection)

            case .zeroReturn: return .closed

                

                

            // Only waiting for a read or write is acceptable, everything else is an error

            case .wantRead:

                

                let selres = waitForSelect(socket: acceptSocket, timeout: timeoutTime, forRead: true, forWrite: false)

                

                switch selres {

                case .timeout: return .timeout

                case .closed: return .closed

                case let .error(message): return .error(message: "...")

                case .ready: break

                }

                

                

            // Only waiting for a read or write is acceptable, everything else is an error

            case .wantWrite:

                

                let selres = waitForSelect(socket: acceptSocket, timeout: timeoutTime, forRead: false, forWrite: true)

                

                switch selres {

                case .timeout: return .timeout

                case .closed: return .closed

                case let .error(message): return .error(message: "...")

                case .ready: break

                }

                

                

            // All of these are error's

            case .wantConnect, .wantAccept, .wantX509Lookup, .wantAsync, .wantAsyncJob, .syscall, .ssl, .bios_errno, .errorMessage, .undocumentedSslError, .undocumentedSslFunctionResult:

                return .error(message: "...")

            }

        }

This accept loop shows something important (that I now realise I did forget to mention in the previous parts... sorry... will update those parts later).

Any SSL_xxx call can result in multiple read/write operations on the POSIX level. Hence for the accept we also need to watch for POSIX read/write events and allow these to occur.

Other than that, accepting is easier than connecting since we don't have to verify the connection manually. The client does that.

To me, this makes sense. A server should be able to work autonomously. But a client can have user interactions where a user might want to accept a certificate that the verification process rejected.

For a server to allow an operator to accept a rejected client certificate is unworkable imo.

So far, everything is fine. The connection is accepted, a session is established and the read/write operations can proceed.

Yes, but... those pesky domains...

In the early days, each server had one domain. But it did not take long to figure out that it is much more cost effective for a server to host multiple domains. And since the domain owners may be unwilling to share a certificate, it must be possible to assign a certificate to each domain.

But the CTX can only handle a single certificate/key combination.

And thus SNI (Server Name Indication) was invented.

SNI takes a little more work. Not only from the server, also from the client. In a normal HTTP request, the client will send the domain name. Unfortunately the HTTP request is something that is done AFTER a secure connection is established. Hence that domain name cannot be used to find out which certificate is needed to start the secure connection.

Thus a client needs to tell the SSL layer (i.e. before the HTTP request is made) which domain it wants to access. Double work...

The client does so by setting up its session. In SecureSockets this is done by this function:

    public func setTlsextHostname(_ name: UnsafePointer<Int8>) {

        SSL_ctrl(optr, SSL_CTRL_SET_TLSEXT_HOSTNAME, Int(TLSEXT_NAMETYPE_host_name), UnsafeMutableRawPointer(mutating: name))

    }

On the server side we need to do some more work.

The SSL handshake that occurs after an SSL_accept call, but before the SSL_accept is finished must have a way to allow the server application to select a certificate.

In OpenSSL 1.1.0 this is done by a callback function. When setting up the context for an accept we need to set the callback destination. Unfortunately this makes a little C-glue code necessary. In C they have implemented some trickery whereby a void(*)() ptr needs to be specified as the callback, but the callback signature itself is Int32(*)(OpaquePointer, Int32*, Void*).

I have not found a way to map those two onto each other in Swift. Hence a little C code:

void sslCtxSetTlsExtServernameCallback(SSL_CTX *ctx, int (*cb)(const SSL *ssl, int *num, void *arg), void *arg) {
    SSL_CTX_set_tlsext_servername_arg(ctx, arg);
    SSL_CTX_set_tlsext_servername_callback(ctx, cb);
}

As can be seen in the installation instructions of SecureSockets, I put this code in some openSSL files. That way I did not need to create any additional libraries, but simply compiled openSSL with this extension in it. I choose the ssl.h and ssl_lib.c files for this. But I think it is rather irrelevant in which files it is placed.

In the callback itself, we need to maintain a list of domain contexts. Each domain has its own context, and that context has the certificate & private key set for that domain. When the callback is activated, it hands us the session as a pointer. From that session we retrieved the asked-for domain name. And then we look for the proper domain context with the right certificate. Once we find the proper domain context, we pull a switcheroo and switch the context of the session to the context of the domain. Bam! done.

In SecureSockets code the installation of the callback and the callback itself are found in the class Ctx:

Installing the callback (calls out to the C-glue code):

    sslCtxSetTlsExtServernameCallback(optr, sni_callback, UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque()))

And the callback itself

    private let sni_callback: @convention(c) (_ ssl: OpaquePointer?, _ num: UnsafeMutablePointer<Int32>?, _ arg: UnsafeMutableRawPointer?) -> Int32 = {

        

        (ssl_ptr, _, arg) -> Int32 in

        

        

        // Get the reference to 'self'

        

        let ourself = Unmanaged<Ctx>.fromOpaque(arg!).takeUnretainedValue()

        

        

        // Get the String with the host name from the SSL session

        

        guard let hostname = SSL_get_servername(ssl_ptr, TLSEXT_NAMETYPE_host_name) else { return SSL_TLSEXT_ERR_NOACK }

        

        

        // Check if the current certificate contains the hostname

        

        if let ctx_ptr = SSL_get_SSL_CTX(ssl_ptr) {

            

            if let x509_ptr = SSL_CTX_get0_certificate(ctx_ptr) {

                

                if X509_check_host(x509_ptr, hostname, 0, 0, nil) == 1 {

                    

                    return SSL_TLSEXT_ERR_OK

                }

            }

        }

        

        

        // Check if there is another CXT with a certificate containing the hostname

        

        var foundCtx: Ctx?

        for testCtx in ourself.domainCtxs {

            if testCtx.x509?.checkHost(hostname) ?? false {

                foundCtx = testCtx

                break

            }

        }

        guard let newCtx = foundCtx else  { return SSL_TLSEXT_ERR_NOACK }

        

        

        // Set the new CTX to the current SSL session

        

        if SSL_set_SSL_CTX(ssl_ptr, newCtx.optr) == nil {

            // The new ctx did not have a certificate (found by source code inspection of ssl_lib.c)

            // This should be impossible since that would have caused this CTX to be rejected

            return SSL_TLSEXT_ERR_NOACK

        }

        

        

        return SSL_TLSEXT_ERR_OK

    }

}

Well, that about wraps things up.

Only one thing left to do ... for you that is... ;-) see below...

Happy coding...

Did this help?, then please help out a small independent.
If you decide that you want to make a small donation, you can do so by clicking this
link: a cup of coffee ($2) or use the popup on the right hand side for different amounts.
Payments will be processed by PayPal, receiver will be sales at balancingrock dot nl
Bitcoins will be gladly accepted at: 1GacSREBxPy1yskLMc9de2nofNv2SNdwqH

We don't get the world we wish for... we get the world we pay for.