Dynamic libraries and code replacements in Swift

Dynamic library packages

I’ve already printed an article about constructing static and dynamic libraries utilizing the Swift compiler, if you do not know what’s a dynamic library or you might be merely a bit extra about how the Swift compiler works, you must undoubtedly check out that publish first.

This time we’ll focus a bit extra on using the Swift Bundle Supervisor to create our dynamic library merchandise. The setup goes to be similar to the one I’ve created within the loading dynamic libraries at runtime article. First we’ll create a shared library utilizing SPM.

import PackageDescription

let package deal = Bundle(
    title: "TextUI",
    merchandise: [
        .library(name: "TextUI", type: .dynamic, targets: ["TextUI"]),
    ],
    dependencies: [
        
    ],
    targets: [
        .target(name: "TextUI", swiftSettings: [
            .unsafeFlags(["-emit-module", "-emit-library"])
        ]),
    ]
)

The package deal manifest is sort of easy, though there are a couple of particular issues that we had so as to add. The very very first thing is that we outlined the product kind as a dynamic library. This may be certain that the precise .dylib (or .so / .dll) binary will probably be created if you construct the goal. 🎯

The second factor is that we would wish to emit our Swift module data alongside the library, we will inform this to the compiler by way of some unsafe flags. Do not be afraid, these are literally not so harmful to make use of, these flags will probably be straight handed to the Swift compiler, however that is it.

Now the supply code for our TextUI library goes to be quite simple.

public struct TextUI {

    public static dynamic func construct() -> String {
        "Hiya, World!"
    }
}

It is only a struct with one static operate that returns a String worth. Fairly easy, besides one factor: the dynamic key phrase. By including the dynamic modifier to a operate (or methodology) you inform the compiler that it ought to use dynamic dispatch to “resolve” the implementation when calling it.

We will benefit from the dynamic dispatch afterward, however earlier than we might transfer onto that half, now we have to construct our dynamic library and make it obtainable for others to make use of. 🔨

In case you run swift construct (or run the challenge by way of Xcode) it’s going to construct all of the required information and place them below the correct construct folder. You can even print the construct folder by working the swift construct -c launch --show-bin-path (-c launch is for launch builds, we’ll construct the library utilizing the discharge configuration for apparent causes… we’re releasing them). In case you listing the contents of the output listing, you must discover the next information there:

• TextUI.swiftdoc
• TextUI.swiftmodule
• TextUI.swiftsourceinfo
• libTextUI.dylib
• libTextUI.dylib.dSYM

So, what can we do with this construct folder and the output information? We will want them below a location the place the construct instruments can entry the associated information, for the sake of simplicity we’ll put every thing into the /usr/native/lib folder utilizing a Makefile.

PRODUCT_NAME := "TextUI"
DEST_DIR := "/usr/native/lib/"
BUILD_DIR := $(shell swift construct -c launch --show-bin-path)

set up: clear
    @swift construct -c launch
    @set up "$(BUILD_DIR)/lib$(PRODUCT_NAME).dylib" $(DEST_DIR)
    @cp -R "$(BUILD_DIR)/lib$(PRODUCT_NAME).dylib.dSYM" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftdoc" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftmodule" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftsourceinfo" $(DEST_DIR)
    @rm ./lib$(PRODUCT_NAME).dylib
    @rm -r ./lib$(PRODUCT_NAME).dylib.dSYM

uninstall: clear
    
    @rm $(DEST_DIR)lib$(PRODUCT_NAME).dylib
    @rm -r $(DEST_DIR)lib$(PRODUCT_NAME).dylib.dSYM
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftdoc
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftmodule
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftsourceinfo

clear:
    @swift package deal clear

Now in case you run make or make set up all of the required information will probably be positioned below the precise location. Our dynamic library package deal is now prepared to make use of. The one query is how can we devour this shared binary library utilizing one other Swift Bundle goal? 🤔

Linking in opposition to shared libraries

We will construct a model new executable software known as TextApp utilizing the Swift Bundle Supervisor. This package deal will use our beforehand created and put in shared dynamic library.

import PackageDescription

let package deal = Bundle(
    title: "TextApp",
    targets: [
        .target(name: "TextApp", swiftSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ], linkerSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ]),
    ]
)

The trick is that we will add some flags to the Swift compiler and the linker, in order that they’ll know that we have ready some particular library and header (modulemap) information below the /usr/native/lib/ folder. We would additionally wish to hyperlink the TextUI framework with our software, as a way to do that now we have to cross the title of the module as a flag. I’ve already defined these flags (-L, -I, -l) in my earlier posts so I suppose you are accustomed to them, if not please learn the linked articles. 🤓

import TextUI

print(TextUI.construct())

Our important.swift file is fairly easy, we simply print the results of the construct methodology, the default implementation ought to return the well-known “Hiya, World!” textual content.

Are you prepared to exchange the construct operate utilizing native methodology swizzling in Swift?

Dynamic methodology alternative

After publishing my unique plugin system associated article, I’ve acquired an e mail from one in all my readers. To begin with thanks for letting me know concerning the @_dynamicReplacement attribute Corey. 🙏

The factor is that Swift helps dynamic methodology swizzling out of the field, though it’s by way of a non-public attribute (begins with an underscore), which suggests it’s not prepared for public use but (yeah… identical to @_exported, @_functionBuilder and the others), however ultimately it is going to be finalized.

You possibly can learn the unique dynamic methodology alternative pitch on the Swift boards, there’s additionally this nice little snippet that accommodates a minimal showcase concerning the @_dynamicReplacement attribute.

Lengthy story quick, you should use this attribute to override a customized dynamic methodology with your individual implementation (even when it comes from a dynamically loaded library). In our case we have already ready a dynamic construct methodology, so if we strive we will override that the next snippet.

import TextUI

extension TextUI {

    @_dynamicReplacement(for: construct())
    static func _customBuild() -> String {
        "It simply works."
    }
}

print(TextUI.construct()) 

In case you alter the important.swift file and run the challenge you must see that even we’re calling the construct methodology, it’ll be dispatched dynamically and our _customBuild() methodology will probably be known as below the hood, therefore the brand new return worth.

It really works like a attraction, however can we make this much more dynamic? Is it potential to construct yet one more dynamic library and cargo that at runtime, then substitute the unique construct implementation with the dynamically loaded lib code? The reply is sure, let me present you the way to do that. 🤩

import PackageDescription

let package deal = Bundle(
    title: "TextView",
    merchandise: [
        .library(name: "TextView", type: .dynamic, targets: ["TextView"]),
    ],
    targets: [
        .target(name: "TextView", swiftSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ], linkerSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ]),
    ]
)

Identical SPM sample, we have simply created a dynamic library and we have used the TextUI as a shared library so we will place our TextUI extension into this library as an alternative of the TextApp goal.

To date we have created 3 separated Swift packages shared the TextUI module between the TextApp and the TextView packages as a pre-built dynamic library (utilizing unsafe construct flags). Now we’ll prolong the TextUI struct inside our TextView package deal and construct it as a dynamic library.

import TextUI

extension TextUI {

    @_dynamicReplacement(for: construct())
    static func _customBuild() -> String {
        "It simply works."
    }
}

We will use the same makefile (to the earlier one) or just run the swift construct -c launch command and duplicate the libTextView.dylib file from the construct listing by hand.

In case you run this code utilizing Linux or Home windows, the dynamic library file will probably be known as libTextView.so below Linux and libTextView.dll on Home windows.

So simply place this file below your private home listing we’ll want the total path to entry it utilizing the TextApp’s important file. We will use the dlopen name to load the dylib, it will substitute our construct methodology, then we shut it utilizing dlclose (on the supported platforms, extra on this later…).

import Basis
import TextUI

print(TextUI.construct())

let dylibPath = "/Customers/tib/libTextView.dylib"
guard let dylibReference = dlopen(dylibPath, RTLD_LAZY) else {
    if let err = dlerror() {
        fatalError(String(format: "dlopen error - %s", err))
    }
    else {
        fatalError("unknown dlopen error")
    }
}
defer {
    dlclose(dylibReference)
}


print(TextUI.construct())

The wonderful thing about this strategy is that you do not have to fiddle with further dlsym calls and unsafe C pointers. There may be additionally a pleasant and detailed article about Swift and native methodology swizzling, this focuses a bit extra on the emitted replacements code, however I discovered it a really nice learn.

Sadly there may be yet one more factor that now we have to speak about…

Drawbacks & conclusion

Dynamic methodology alternative works good, this strategy is behind SwiftUI reside previews (or dlsym with some pointer magic, however who is aware of this for positive..). Anyway, every thing seems nice, till you begin involving Swift lessons below macOS. What’s unsuitable with lessons?

Seems that the Goal-C runtime will get concerned below macOS in case you compile a local Swift class. Simply compile the next instance supply and check out it utilizing the nm device.

// a.swift
class A {}

// swiftc a.swift -emit-library
// nm liba.dylib|grep -i objc

Below macOS the output of nm will comprise traces of the Goal-C runtime and that’s greater than sufficient to trigger some troubles in the course of the dylib shut course of. Seems in case your library accommodates the ObjC runtime you will not have the ability to really shut the dylib, it doesn’t matter what. ⚠️

Previous to Mac OS X 10.5, solely bundles may very well be unloaded. Beginning in Mac OS X 10.5, dynamic libraries might also be unloaded. There are a few circumstances during which a dynamic library won’t ever be unloaded: 1) the principle executable hyperlinks in opposition to it, 2) an API that doesn’t help unloading (e.g. NSAddImage()) was used to load it or another dynamic library that is dependent upon it, 3) the dynamic library is in dyld’s shared cache.

In case you check out man 3 dlclose you will get a couple of extra hints concerning the causes, plus you too can examine the supply code of the Goal-C runtime, if you wish to see extra particulars.

Anyway I believed this needs to be talked about, as a result of it could actually trigger some bother (solely on macOS), however every thing works simply nice below Linux, so in case you are planning to make use of this strategy on the server aspect, then I would say it’s going to work simply high quality. It isn’t protected, however it ought to work. 😈

Oh, I virtually neglect the hot-reload performance. Nicely, you may add a listing or file watcher that may monitor your supply codes and if one thing adjustments you may re-build the TextView dynamic library then load the dylib once more and name the construct methodology if wanted. It is comparatively simple after you’ve got tackled the dylib half, as soon as you determine the smaller particulars, it really works like magic. 🥳