Archive for category Xcode

XCTest with Xcode 6

These are my notes from the 2014 WWDC Session 414 Testing with Xcode 6

What is covered

  • Benefits of testing
  • Getting started and adding tests to old projects
  • Asynchronous testing
  • Catch performance regressions


Why Test?

  • Find bugs
  • Hate regressions
  • Capture performance changes
  • Codify requirements – tests help to codify the range of functionality of code.
  • The tests themselves will tell other engineers what expected behaviour is.
Add tests to an existing project
In a new project
  • You can write tests first
  • Write code that passes those tests
  • This is known as Test driven development

Continuous integration workflow

  • Start off in a green state
  • Green state represents known quality
  • Add features, fix bugs, add new tests
    • At some point a test will fail
    • This will be flagged immediately

Xcode testing with Framework XCTest

  • Expectations passes/failures
  • To create tests subclass XCTestCase
  • Implement test methods
  • Return void, method name starts with “test”. Rest of name should describe purpose of test
    • -(void)testThatMyFunctionWorks
    • Use assertion APIs to report failures
    • XCTAssertEqual(value, expectedValue); // Compares two scalar values
    • If a failure, outputs a string and reports a failure to the test harness.
    • Xcode has test targets for managing tests.
  • Test targets build bundles
    • Contains the compiled test code
    • Resources you use in the test
    • These go in the test bundle, not in the application
    • Test targets are automatically included in new projects
    • You can have as many test targets as you want so you can break tests up into groups.
  • Test bundles
  • Test hosting
    • Tests are hosted in an executable process
    • The tests are usually injected into your app.
    • That means the tests have available all the code in the application.
    • Alternatively you can run the tests in an hosting process provided by Xcode
    • When you go to load resources for tests the resources are not in the main bundle
      • Don't do [NSBundle mainBundle]
      • Instead: [NSBundle bundleForClass:[MyTest class]]
  • Running tests
    • Simplest way is Command-U
      • Takes the active Scheme and runs the tests for that scheme
    • In the editor window gutter there is also run test buttons. You can run a single test, all tests in a test class
    • There is a similar set of buttons in the test Navigator
    • You can also run tests using xcodebuild
      • You can create your own automation setup
      • xcodebuild test -project ~/MyApp.xcodeproj -scheme MyApp -destination 'platoform=iOS,name=iPhone'
    • Where are the results displayed
      • The test navigator where you’ll have green/red checkmarks against each test
      • In the issue navigator
        • You’ll get the failure, and the reason for the failure.
      • In the Source editor gutter.
      • In the test reports, which show all tests that are run and associated logs.
    • Demo – Add tests to an existing project
    • One major point. Keep tests simple so it is clear why the test failed.
      • E.g. If your testing parsing data from the internet. You don’t want to test internet access.
      • Download & save the data and add the saved data to the test target.
      • Then load the data from a file.
  • Asynchronous Tests
    • More and more APIs are asynchronous
      • Block invocations
      • Delegate callbacks
      • Make network requests
      • Background processing
    • Unit tests run synchronously so this creates a challenge
    • XCTests adds the “Expectation” objects API to Xcode 6 testing
      • The expectation object describes events that you expect to happen at some point in the future
      • - (XCTestExpectation *)expectationWithDescription:(NSString *)description
    • XCTestsCase waits for expectations to be fulfilled.
      • - (void)waitForExpectationsWithTimeout:(NSTimeInterval)timeout handler:(XCWaitCompletionHandler)handlerOrNil
      • This will wait until the timeout interval is complete or when all events have been fulfilled
      • Testing opening of a document asynchronously:
- (void)testDocumentOpening
  XCTestExpectation *expectation = [self expectationWithDescription:@"open doc"];
  IDocument * doc = ...;
  [doc openWithCompletionHandler:^(BOOL success) {
    [expectation fulfill];
  [self waitForExpectationsWithTimeout:5.0 handler:nil];

See the demo code for writing an asynchronous test. Basically look for earthquake parser.

Performance testing

  • It can be easy to introduce performance regressions in code.
  • Catching these regressions is difficult.
  • Performance testing allows you to automate this
  • New APIs to measure performance
  • New Xcode UI to interpret results
    • Can profile tests with instruments
    • Use the new measure block API `- (void)measureBlock:(void (^)(void))block;`
    • Takes a block of code and runs it 10 times and shows the results in Xcode


  • See the demo code for testing performance. This is the mac version of the eartquake parser.
    • Call -measureBlock: to detect performance regressions
    • View results in Source Editor and Test Report
    • Profile tests with instruments.
    • Setting Baselines – Needs a fix point to compare against
    • Standard deviation – How much variation is allowed before a performance is considered to have failed
    • Measuring precisely
    • Baseline is the Average from a previous run
      • Set Baseline Average to detect regressions
      • Fail if > 10% increase from Baseline Average
      • Regressions less than 0.1 seconds are ignored – to remove false positives
    • Baselines are stored in source
    • Baselines are per-device configuration
    • Include device model, CPU, and OS


Tags: , ,

Git submodules

At this point I’ve decided not to go down the Cocoapods route to manage my project dependencies. This has come up now because I’ve added the first external code to my project and the installation instructions was basically use Cocoapods.

I’m no Cocoapods hater, just that it adds another layer of complexity that makes things easier when it works but adds one more step that can wrong with the toolchain. If I decide to use Cocoapods I will make the effort to properly understand how it works so when it does go wrong I’m not scrabbling around in the dark.

So after a recommendation from @abizern to try git submodules I’ve decided to give that a go.

The external code is a logging framework called CocoaLumberjack.

At the top level of my git repository I want to create a folder called CocoaLumberjack which is where the submodule will live.

git submodule add git:// CocoaLumberjack

The above command creates the CocoaLumberjack folder and downloads the current state of the CocoaLumberjack project. It also creates a “.gitmodules” file if you’ve not previously added a git submodule to your project and adds an entry for the CocoaLumberjack submodule. It then adds the CocoaLumberjack folder to your own repository, this will not add the submodule files to your git repository, and then adds or update the “.gitmodules” file.

After pushing everything to your remote, other clients of that remote should do a git pull like normal which will create the folders for the submodules but not check them out. Within the folder for each submodule you should run the commands:

git submodule init
git submodule update

Obviously if the CocoaLumberjack project is updated and you want the latest revision then git submodule update will need to be run at the top level in each repository.

You can update all submodules at once if needed using the following command when you are in the top level folder of the repository containing the submodules.

git submodule update --init --recursive

I can now go and add the necessary files to my project and away I go.

From Abizer’s comment below, to clone a remote repository which contains submodules you can do the following:

 git clone –-recursive

I’d also recommend reading his blog post that he linked to.


Tags: , ,

Unit and system testing

I’ve been way behind the curve with looking into whether I should be going down the unit test approach.

I’ve finally started looking into what approach to use for testing during the process of development for MovingImages that will work best.

My first efforts at finding out about using test driven development and Xcode 5 were frustrating. Xcode 5 comes with XCTest but there is very little documentation in relation to unit testing and Xcode 5. I found some Apple sample code which demonstrates the use of unit testing, unfortunately it uses SenTestingKit which was Apple’s former unit testing framework that has been replaced by XCTest in Xcode 5. There is a migrator tool that you can use to move your pre Xcode 5 testing code to Xcode 5. But still not much help with documentation.

Read the rest of this entry »

Tags: , ,

Codesigning, Xcode 5 and Mavericks

Code signing in Xcode 5 has changed and is problematic.

This is about signing Mac Applications for distribution outside the Mac AppStore and meeting gatekeeper requirements. I’ve not done anything with entitlements.

There has been some discussion on this on the Apple developer forums.

Some notes as a conclusion was reached have been published on this blog here.

A useful command is:

spctl -a -vvvv Path/To/App/

The code signing identity that needs to be used is Developer ID Application: K Meaney (U6T…).

I’ve applied that developer id at the project level and allowed each of the targets to inherit the code signing identity except the test target which has had the code signing identity scrubbed.

I had problems that my mac developer id was behaving like it was generated from the iPhone provisioning profile. I couldn’t work out from the many key chain identities/certificates/public and private keys which one it was that was causing the problem. So I scrubbed all the developer related keys/certificates/identities from my keychain. Then in Xcode in the accounts section of the Preferences I clicked View details, then clicked on the Mac Team Provisioning Profile then clicked on the + symbol and selected “Developer ID Distribution” which then created a “Developer ID Application” signing identity. It is this signing identity that I used to code sign moving images.

The Validate button in the organiser window is pointless for my purposes as it won’t deal with application or any package that contains multiple bundles. That is why the spctl -a -vvvv command is useful as it allows you to test that you have code signed your application successfully.

The last issue I had to deal with was that Apple’s timestamp service is often down. There is no sensible array around this.

Gatekeeper, codesign, code sign, codesigning, code signing.

Tags: , , , ,

Adding new dependencies

Since this has stumped me at least twice now, wasting over an hour each time trying to work out what the solution is, I decided to write it down.

If you have a Xcode workspace with a couple of projects in it, how do you add targets of one project (project 1) to be dependencies of a target in a different project (project 2) all within the same workspace. The answer is that you add the dependencies in the scheme editor associated with the target in project 2, you do this in the Build action section of the scheme editor. The find implicit dependencies option in the scheme editor doesn’t always work across projects. Whatever the setting for the implicit dependency option, I like to make my dependencies explicit.

I added the products which belong to project 1 to project 2 for inclusion in the application bundle, but the copy files build phase didn’t find them. Two changes were necessary for the project 1 products to be found by the project 2 target build. I had to get the build order of the dependencies and the final product correct in the scheme editor, and secondly I had to change the path for project 1 build products to be relative to the group. I don’t know why the copy files build phase couldn’t find the project 1 build products before making this change. But after making these changes it worked.

Tags: , , , ,