Instruments Automation screenshot ignores programmatically added controls

Instruments Automation screenshot ignores programmatically added controls

I am saving a screenshot in Instruments Automation, but some controls are missing in that screenshot. It looks like programmatically added controls are "ignored".

How can I fix that?

Manual screenshot in Simulator: (note the yellow box)

Screenshot in Instruments Automation:

The automation script:

var target = UIATarget.localTarget();  target.delay(0.5)  target.captureScreenWithName( "screenshot1.png" );  

I created a new single view application in Xcode (universal, objective-c). I added a button and a label with some constraints for auto layout in the storyboard.

I added this code to add the yellow button programmatically:

- (void)viewDidLoad {      [super viewDidLoad];      // Do any additional setup after loading the view, typically from a nib.        UIButton *b = [[UIButton alloc] init];      b.backgroundColor = [UIColor yellowColor];      [b setTitle:@"Extra" forState:UIControlStateNormal];      [b setTranslatesAutoresizingMaskIntoConstraints:NO];      [self.view addSubview:b];      [self.view addConstraint:[NSLayoutConstraint constraintWithItem:b attribute:NSLayoutAttributeTop relatedBy:NSLayoutRelationEqual toItem:self.view attribute:NSLayoutAttributeTopMargin multiplier:1.0 constant:0.0]];      [self.view addConstraint:[NSLayoutConstraint constraintWithItem:b attribute:NSLayoutAttributeTrailing relatedBy:NSLayoutRelationEqual toItem:self.view attribute:NSLayoutAttributeTrailingMargin multiplier:1.0 constant:0.0]];        [self.view setNeedsUpdateConstraints];      // Correction: - initially I used the line below, but that was wrong      // The problem is not solved with using setNeedsUpdateConstraints.      // [self.view updateConstraints];  }  

I'm using Xcode Version 7.1.1 (7B1005), my OS is El Capitan 10.11.1 (15B42). I know that it worked in earlier versions, because I use ui-screen-shooter for my app's screenshots and the problems occur now and it worked before. It is not a problem with ui-screen-shooter, because I can reproduce it with instruments automation alone.

What can I do about it?

EDIT:

One difference between controls in storyboards and manually created ones might be the object ID, which one can see in the storyboard source. In case this would be the problem, can I set the id for manually created controls? (Or is it lost, once the scene is read from the storyboard?)

EDIT 2:

I fetched all properties of the elements in the view controller's view (via objc/runtime) and compared the property values of the two buttons. There are little differences: (won't include the identical entries)

  Button from Storyboard           |  Manual Button  -----------------------------------------------------------------  "_defaultRenderingMode" = 2;     |  "_defaultRenderingMode" = 1;  text = Button;                   |  text = Extra;                                   |          (several position values slightly different)                                   |  description contains:            |  "autoresize = RM+BM;"            |                                    | backgroundColorSystemColorName = yellowColor;  

EDIT 3:

A screenshot of the output of target.logElementTree():

EDIT 4:

I added

[UIButton appearance].backgroundColor = [UIColor yellowColor];  

and all buttons now have the yellow background, however, the "Extra" button is not in the screenshot.

I'm not primarily focused on custom appearance. I also tried

UIButton *b = [UIButton buttonWithType:UIButtonTypeSystem];  

so the button is just like the typical blue buttons. Still the button does not appear in the automation screenshot.

EDIT 5:

In case it is relevant: I can access the controls from the automation script and make taps - for instance (from another app, not this test code):

target.frontMostApp().mainWindow().buttons()[25].tap()  

The taps are processed, so the automation script can access the controls.

EDIT 6: I created a bug report at Apple's bugreport system. If you can reproduce it, it might be good to do the same (at least that's how I understood the intended use of the bug reporting at Apple).

EDIT 7: (thanks to quellish for your answer - processing my first wave of thoughts here:) I corrected an error in the code - I called [self.view updateConstraints];, but it should be [self.view setNeedsUpdateConstraints];. That however, had no influence to the automation screenshot results.

I was wondering, whether constraints were updated and logged calls to updateViewConstraints and some others. This is how it is logged while the app is loading:

viewDidLoad  viewWillAppear  updateViewConstraints  viewWillLayoutSubviews  viewWillLayoutSubviews  viewDidAppear  

So when I add the button and the constraints in viewDidLoad, and updateViewConstraints is called, I would assume that all constraints are settled when viewDidAppear is called.

I made a quick test using this in viewDidAppear: (Not yet sure, whether this is the way to go - I have little experience with accessibility.)

_extraButton.accessibilityFrame = UIAccessibilityConvertFrameToScreenCoordinates(_extraButton.frame, self.view);  

The automation screenshot remains white.

It is interesting however, that in the element tree the screenshot portion of the extra button shows the correct location. So I would assume, that automation picked up the correct position.

Answer by quellish for Instruments Automation screenshot ignores programmatically added controls

The problem you are seeing is because of several factors. As stated in your question, your issue is that:

• UIAutomation screenshots will not include your styled button given the test project you have included and the automation script.

Under different conditions you see different behavior:

• When the button is instantiated through a nib or storyboard the issue does not occur, only when it the button is declared and added to the view hierarchy in code (specifically, viewDidLoad of the view controller).

• The button does appear in UIAutomation screenshots when run in earlier versions of iOS (i.e. iOS 8).

That said, if you remove the constraints added in viewDidLoad you will see the styled button start to appear in screenshots. This should be a hint - the constraints are part of the problem.

When the layout engine solving constraints it attempts to do as much work as possible in batches, coalescing constraints for performance. When the button and constraints are described in a nib or storyboard the constraints engine will get them all at once, and much earlier in the view hierarchy's life cycle. Because of this the steps of the layout process can be streamlined significantly.

When a constraint is added later in the view hierarchy life cycle however, the constraints need to be re-evaluated. Think of the added constraints as being added to a queue and the layout engine will re-evaluate them the next time the queue is "full". UIAutomation hooks into the UIAccessibility APIs to do it's thing - and accessibility may not know or care about what is happening to the layout of the visual representation. The visual representation may change, but not in a way that affects accessibility, and thus UIAutomation's view of the world.

iOS 9 changed several things about how the layout process works under the hood, which is why you are seeing this issue now.

There are a number of courses of action you can take:

• Build your view hierarchy in a nib or storyboard. This will get you the most consistent experience and has performance and other advantages.

• Use the accessibility API to hint that the position (accessibility frame) of the button has changed after the layout pass. Unfortunately you are still likely to see UIAutomation capture it while layout is occurring unless accessibility, visibility, or user interaction is disabled until layout is complete.

• Instead of an arbitrary delay in your automation script, use the UIAutomation timeouts effectively to wait for the element to become available. This is covered in the documentation and is a best practice. Make it unavailable or invalid until layout has completed.

Using these methods I was able to easily get your example project to not reproduce the problem you are seeing. Some, or all of these solutions may be appropriate for your project - or not.

Fatal error: Call to a member function getElementsByTagName() on a non-object in D:\XAMPP INSTALLASTION\xampp\htdocs\endunpratama9i\www-stackoverflow-info-proses.php on line 71