Enhancing User Experience with ARKit for a Furniture Manufacturing App

Enhancing User Experience with ARKit for a Furniture Manufacturing App

Augmented Reality (AR) gives us the ability to enhance our world with a real-time overlay of sound, graphics or video. With the promises of Swift’s ARKit, developers have the opportunity to harness this great power and create projects where our only limitation is the imagination. We had the pleasure of utilizing augmented reality and specifically Apple’s ARKit in a recent application for an American furniture manufacturing company. This client wanted to step into this innovative space and turned to Shockoe to create a proof of concept that takes their existing system for documenting defects with a deck of cards and digitize it with an AR app to enhance this process. Let’s dive into more details on this exciting project.

The Client

Our client is a major manufacturer of quality kitchen and bath cabinets for the home with over 500 cabinet styles and a variety of designs, materials, and finishes. You can find these cabinets being sold through a nationwide network of homebuilders, independent dealers, and distributors. In the U.S., many of their products are found at Lowe’s and The Home Depot. So how can a corporation that boasts the embodiment of the marriage of technology, automation and American craftsmanship benefit from an AR experience?

The Ask 

Our team was tasked to create a digital field tool that can aid in the assessment and documentation of product defects using augmented reality and QMS standards. What were we working with? Well, interestingly enough, the current system for assessing a product defect was a deck of transparent cards that employees would hold up against a cabinet surface to mark the defect. These cards contained instructions and details that would allow an employee to properly mark a defect on the appropriate surface. Not only was this process inefficient, it was also ineffective and gave the chance to human error to the user experience. We had to make a change.

The Build

In order to encapsulate these deck of cards, we started with user testing to begin to understand the user mind. We first provided the user with options of cards to choose from in order to provide the appropriate marker for the defect. These options included species, finish and joint type which were used to drive the next list of defect options. Once each category had an option selected, the user was now able to document the defect. You can see examples of this in these screenshots.

AR-app-manufacturing-AR-assesement-screen

Based on the option selected, the user was given a different spec and AR type used to mark the defect. Specs ranged from 1/4” width x 8” length box to parallel lines .010” in width. Precision was key when marking these defects.

Using ARKit, we are able to create virtual objects with these exact specs to best represent the marker. One of the most important benefits of using AR is that it maintains proper scale from any perspective/distance (which is why we couldn’t simply put an overlay over a photo). This killer feature of AR makes it the right feature in this particular case. ARKit was the ideal technology to solve the client’s problem, and we happen to have the skills to do it.

Here is the code I used to map out these options.

func createBoxImage(length: Float, width: Float) -> VirtualObject {
		
	let position = SCNVector3(x: 0, y: 0, z: 0)
		
	let distanceY = self.get3DPositionInYFromPointForDistance(inches: width, fromPosition: position)
	let distanceX = self.get3DPositionInXFromPointForDistance(inches: length, fromPosition: position)
		
	let scnBox1 = SCNBox(width: distanceX, height: 0.001, length: distanceY, chamferRadius: 0)
		
	let rectangleMaterial = SCNMaterial()
	rectangleMaterial.diffuse.contents =  imageLiteral(resourceName: "focus_square.png")
	scnBox1.materials = [rectangleMaterial]
		
	return VirtualObject(geometry: scnBox1, modelName: "Box \(length) in x \(width) in")
}

 

With  ARKit, we had access to vertical planes allowing us to detect and place nodes on vertical surfaces. Once the vertical surface was detected, we prompted the user to tap and place the marker on the defect. The user also had the option to rotate or drag the marker appropriately once placed.

As the last step and to further enhance their experience and provide better documentation, we’ve added the ability to take photos of the defect as well as take notes about the current defect.

The Outcome

Using ARKit allowed us to provide multiple tools into one mobile app that are efficient and innovative. Users are able to access data easily in the field, improving training and speed of proficiency. By increasing speed, accuracy, and efficiency, all business units will be able to operate with more objective assessments of business KPIs. Augmented reality allows for the automatic surface detection and reliable and accurate scale that decreases the level of subjective user assessment. We’re excited to see the progress of this furniture manufacturing company and help clients with similar needs. Reach out to us if this sounds like the right solution to help your team or if you have your own idea of utilizing augmented reality for your app!

Shae Hazelwood

Shae Hazelwood

Developer

Shae is a mobile developer with a passion for producing business-focused applications and strategy-based mobile development. What began with home-grown app prototypes has turned into a career at Shockoe focused on Titanium and Android development as well as considering market deployment practices. When he’s not deep in code, Shae spends his time playing video games and listening to Hip Hop.

A Mobile Workforce: What Customers Want

A Mobile Workforce: What Customers Want

Preface with a confession

Full disclosure: I am not a developer nor a designer — I’ve been in business ops for most of my professional career. That said, I’ll give myself a dose of credit: I’ve been in tech for years, start-ups and well-established alike. During that time I’ve seen that when it comes to mobile, one thing is consistently fueling a successful app — whether aimed at mobilizing a workforce or driving loyal customers back, apps must always be ready to provide utility in a moment of need. No exceptions.

When I truly needed mobile

This morning I woke up at 5 to the sound of torrential rain. While usually, I would dig myself deeper into my blankets, today it forced me out of bed at a much earlier hour. As it turns out, the night before, my wife detected some water coming into the basement, and I had a feeling it was not going to be a pretty sight downstairs.

I grabbed my phone to kick the lights on around the house. That’s when I noticed my phone was at 75% charge. Strange. It was vexing enough at such an early hour, that after swiping around for a few tired seconds, I forgot to turn the lights on.

I stubbornly walked in the dark and guided myself through the house by carefully feeling out the usual milestones — edge of the bed, door frame, banister, and finally the kitchen light switch. “Damn!” The power was out, and now I had a soaked basement and no light to resolve it.

Be the tool customers want when things fall apart

When things start failing at home, people like myself try to find help in the quickest, cheapest, and most accessible solution possible. For the water, it took shovels, mops, and wheelbarrows. As for the power outage, I had to go through a web-portal, find a billing statement to get an account number, and finally after going through some more personal identification prompts, five minutes later, I was reassured to see a team had been dispatched to fix the problem.

Keeping demanding customers like myself pleased takes some finesse. While I have to give props to my utility company for giving me a five-minute solution, the rising water was not exactly being resolved as I was searching for web-access.

Going mobile is tricky but very doable

Deploying the kinds of mobile management tools that customers crave sometimes go hand-and-hand with the tools that can enable a workforce to act faster, give prompter updates, and resolve more issues in less time. Yet, if these tools don’t exist, it’s alright. Rome wasn’t built in a day after all.

At Shockoe, we’ve done everything from overhauling pen & paper methods and building mobile ecosystems from scratch, all the way to optimizing legacy systems and tapping into API’s to deploy better user experiences. That said, we believe there is a right way to move from antiquated systems, and a measured means of building out tools that can improve your overall service, empower your workforce, and ultimately give customers with flooded basements, the 30-second solution they truly needed.

Step one: Low Investment High Return

If you’re starting from scratch, start internally. Typically your team will be more forgiving than your customers. Figure out what information has been missing as technicians work in the field to complete tickets, receive orders, or communicate with HQ. Regardless of what it is, keep your vision clear, uncomplicated, and unobstructed. Most importantly, keep your metrics measurable and your data visible in a way to demonstrates efficacy to your deployment.

Here are a few things we’ve seen as a valuable step one for Mobile Workforce applications:

  1. Route Mapping: Optimize the travel path between stops by providing real-time directions and automatically tracking time at each stop.
  2. Messaging and Feedback systems: Enable technicians to send notes, photos, and videos back to supervisors or other technicians for feedback, advice, and training.
  3. Repair/Service Supplies: Enable technicians to keep track of inventory and supplies riding with them on the truck. When technicians use items, build an app that keeps track of how much. And when supplies, run low, have the system push a notification (I encourage reading our node.js blog post on how to best do this) to restock the next time they’re parked at HQ.

The key to step one is to make sure you get buy-in from your team by delivering a product that truly saves time and delivers the utility that encourages them to use it time and time again.

Step two: Build on success, Invite your Customers Along

Necessity drives innovation. A tool that is simple will eventually reach a threshold where it must either be built out further or run the risk of feeling limiting to your team. Build on what works, and start giving access to your customers to tap into the data that is relevant to them. I can’t stress this enough — keep utility front and center. Flash, gimmicks, shameless marketing will be evident in a heartbeat. If you want your customers to stay loyal, give them genuine utility and a true reason to pull their phone out when the house is slowly flooding.

A few stage two deployments we encourage:

For driving a mobile workforce

  1. Gamification: create dashboards and performance metrics that show customer satisfaction, ticket time to resolution, accuracy, and safety performance.
  2. AR & VR Training: In some industries, training can entail a safety risk. By incorporating emerging visual technology into your training program, a technician can learn a new skill in a safe, controlled environment.

For customers with midnight woes

  1. Deliver utility first: payments, statements, reporting outages, leaks, and other common service hiccups. The time for your company to shine is when things go awry for your customers
  2. Deliver details: The same system powering technicians can now give a more granular view of when things will be repaired. If you really want to go the extra mile, add an option for customers to opt-in to push notifications giving regular updates to when crews are dispatched, assessing damage, in repair mode, and of course, when repairs are completed.

I get it, it can be daunting to jump into mobile. People are often held back by clunky systems, outdated back-ends, lack of expertise, or general fears of delivering a poor application. But just like the flood, what started with a pushbroom and a lot of cursing, turned into a full scale operation with flashlights, shovels, ladders, and a lot of elbow grease that ultimately cleared out the basement, and fixed the issue at the root — I’ll listen next time my wife tells me to clean the gutters before the spring-time truly rolls in. I would encourage companies to listen to their customers in the same way.

It takes a clear vision and a well-constructed roadmap to take a company from antiquated systems to cutting edge — but don’t fear, it’s doable! If you ever need a hand, you know where to find us.

— Jaime De La Ree is the Business Development Lead at Shockoe with five years of experience in mobile technology sales