Dashboard Prototype Test 3

  • Participant: Vesa
  • Fascilitator: Ilya
  • Observer: Nazia
  • Date: 19.06.2017

Observations:

(wireframe1) Question 1:what do you tell about the dashboard in general?

  • Participant sees a quick looks to familiar APIs.
  • Participants sees average response time, no. of calls and unique users. Everything the participant wants to see in the first place. He doesn't find anything missing that he'd want to see

(facilitator explains what the dashboard UI represents) Question 2:Would this be enough for you or not? - Participant can't think about anything missing from. He would be very happy to start with this.

Question 3: How do you think the elements are working here? do you know where the numbers come from and what they stand for? - Participants thinks the numbers come from the API solution. It shows how many times the API has been called, the average response time of the calls and how many developers have used it.

Question 4: You come in this view and see it as week (indicating the time period). What does it mean to you? - participant was wondering if overview was for last week or past seven days.

Question 5: About the elements of visualization here, does the numbers displayed here are enough for you? Or you want a sparkline chart or any sort of chart? - participants find the numbers to be enough, specially with the percentage amounts to indicate how much a metric value have increased or decreased in the past week.

Question 6: what you like about the design(1st wireframe)? - Participant finds it simple - He finds the main data presented. - The numbers are enough for the participant to know if anyone uses his API, how the usage has changed. There is nothing else he can think about how he can measure his APIs.

Question 7: what about monitoring and performance? what information would you like to see in addition to these numbers? (e.g. performance issues, downtimes, any problems, etc) - The downtime might be useful to see. The view (1st wireframe) generally portrays that everything is fine here.

Question 8: Would you like to see here if your API has a problem? - The participant would like to see an indicator if an API has a problem. specially if it has a stability issues. This UI gives an impression that everything is great, which might not be the case

Question 9: Would you like to see somehting more here in this dashboard, like a detailed view? What you expect if you click on any of the mertrics or anywhere? - Participant doesn't know what might happen. He thinks he'd again get some overview about his API. He is unsure whether or not there would be any specific statistics about unique users. Participant assumes that the geenral action would be he gets in depth about information on his API.

Question 10: (facilitator shows the second wireframe and tells its purpose) What do you think about it in general? what about these 2 charts and numbers - participants is a bit disappointed and asks is that all what he gets in the details view. - participant things the details gives exactly the same thing as overview andf nothing new.

Question 9: Is the duplicating information is useful to you? - Participnat feels the information is useful. The number of calls in detailed view would be more helpful, if it live. If this is shown as ahistorical data, it feels to participant that something is hidden.

Question 10: (observer) Why you need live data? - If the details is run live, then there is meaningful information for the participant in the displayed charts. If it is deeper analysis online, specially with a reload button, he's like to get all possible data for the api

Question 11: How useful it is for you to look back over historical data from a chart? - as a APi owner, participant wants to know if there is any spike, when the spike has come, why there is peak need of my API, when my APi has been most popular. - Participnart wants to look back at time and want to see if he has reached some limits, some special cases that have happened.

Question 12: So you want monitoring feature to know what is happening with your API (e.g. downtime, when it was slow? - Participant is not much of an API owner. But if he's running a company that is building APi, he may want to know anbout downtime to understand if customers are making complaint against it. But this won't happen if downtime is not shown. - participant'd like to know if my company is having problems with the backend of an api.

Question 13: From the charts, number (and pecentage) you get to see where has been some rise and fall of number of calls or response time and they are not average. What is your next step? - Parcipant may be navigating from details to somewhere else to know what is happeneing. - participnat might want to predict when the next instance the abnormal rise7fall of any metrics may hapopen. - this may need some learning about the API usage for the platform. - If an API is having a large response time and participnat want to know why, the platform might not be able to fix it. - If the participnat is doing API business, and an API getting a lot of calls, the business is right. The reason should be gifured out somewhere esle, not in the platform. - On the otherhand if a backend is down, he can't fix it in the platform. the dashboard should be working as a messenger to inform the participant about it. - However, if there is broken user management, down proxy or broken API keys, dashboard should let him know about these.

Question 14: (facilitator shows the 2nd version of wireframe) Here informaton about bookmarks, average rating, numbers with percentage and graph are shown. What do you think about this? - participants find the information too much - (in comparison with 1st wireframe)For each API, there is too many things to look at the 1st glance. - This makes u stop and think what is a particular information about and how does it map with other elements, even with other APIs. - the graphs don't add anything special or are informative in temrs of information. - Participnat doesn't like too many graphs in the same place.

Question 15: (facilitator shows the 2nd version of details view) Here the details view shows some combined problems of the APIs. (like https timeout, downtime, etc.) is it useful to you? - This is useful for the participnat. As he is taking care of his customers he need to know about the problems that are arising. - The developers using his APIs get affected with the problems, so thi is important for him to know.

Question 16: If there is a list of problems given in detailed view, would you like to prioritize over the graphs? perhaps stating them before the graphs are shown? - If there are some serious problems, (e.g an API is unreachable to developers over the week) participant would like to know about it immediately.

Question 17: For monitoring part, you want to spend as little time as possible to understand if everything is OK? How much time would you like to spend in dashboard to understand this? - When participant starts his APi, he wants to see how popular it grows among users - When developers are using the API and starting to make more calls, participant wishes to know if they are facing problems, what problems they are facing. Also participant wants to know how happy the customers are.

Question 18: (Observer) How do you wish to see the users? specific users (id, email, api keys) making call to your API? Or actual services using your API? - If Participnat has some unique APIs, he might want to see who follows it. May be categorize the users based on number of calls made (heavy users).

Question 19: (Observer) Is seeing historical data as week based trends is enough for you? Or would you prefer both weekly and monthly? - For participant, it is enought to see the trend in weekly basis.