Dashboard Prototype Test 2

  • Participant: Jarkko
  • Fascilitator: Nazia
  • Date: 15.06.2017

Observations:

(wireframe1) Question 1:what do you think about the design?

  • Participant asks how he can know which API needs his immediate attention. (mainly indicating the entire API or specific matrics under an API)
  • Participants wishes to know how he can filter the APIs displayed (e.g. searching for a specific API, a slow or down API)
  • Participant wished to know how he can have the problematic APIs appearing on top of the list, or its there any way to configure it.
  • Participant asked how the refresh button works. He preferred for automatic refresh or atleast some checkbox to ensure this instead manusl refreshing.
  • Number of failed calls are more important to partcipant. He wishes to see this information as well
  • Participant also wishes to see a quick number or texts about number of APIs that need his immediate action before seeing the overview.

Question 5: what do you think happens when you touch a card here? - on clicking a card, it gives the participant a detailed overview of the selected API (we switched into the second wireframe) - participant wishes it to open the detailed view in a new tab, so that he can swich back and forth between dashboard and that specific API. It also lets him to use the url and give to developers to show where the problem is - Number of calls and Response time seems satisfying to the participant. However, he wishes to see the correlation between response time and slow/failed calls in the charts. (e.g. at some point API response time rises as a spike and there had been a failed call/slow call. Participant wants to show indicate/refer this to dig deep) - Participant also asked how refreshing worked there.

Question 8: what you like about the (1st wireframe)? Are the shown information enough to know about your APi statuses? - As a generalized view about APIs the participant sees, the information is sufficient to know about the status. - For unique users, it is sufficnent that the users are tracked using API keys or similar technique. The participant at this moment is not worried about the specific services.

Question 9: what improvements would you like to see there (1st wireframe)? - Participant is in more favor of seeing Errors instead of number of unique users. He's rather see errors (e.g. failed calles, etc) instead of errors. - Participant wants indications about things that should have in attention. If cards are all in same tone, he might not be figure out about which APIs he should pay attention to. - As a part of customization, participant would like to have one or multiple APIs to monitor as sticky. That is they should always appear on top. This is helpful when a new API is published and is needed to monitor to know how it performs on the fly.

Question 10: what you like about the (2nd wireframe)? Are the shown information enough to know about your APi statuses? - yes, may be.

Question 10: what improvements would you like to see there (2nd wireframe)? - Participant wishes to see a list of slow/faild/error calls and their time for a parcitular APi. Those can be endpoitn specific. And also can include the particular path and method for which, the error calls are happeneing. This helps to correlate the Number of Calls chart and Response Time charts in order to trace the error call. - Participant wishes to see failed/error/slow calles in the number of call charts - Participant might like a Monthly view that he sees occassionally to find out APi usage pattern. - to detect API hard problems (e.g. performance degrade), details such as HTTPS handshake durations, DNS lookup duration, etc. might be needed.

Question 11: Please give your feedback with this (2nd version) of Dashboard? - Participant's 1st impression of the wireframe is that it looks too crowdy - It is hard to see and understand which numbers and visualization is related to each other and how? - Participant emphasizes more on the numbers and they should be the 1st thing to attract his attention - The small trends graph can be used if they don't compromise the readability. - For participant, the numbers should be the main thing to focus first in the 1st glance. The trends are optional. There is always a choice between percentage with up/down arroaw and small trend charts. - The rating information and bookmarking information are not needed in dashboard. They can be in the details view. - Participant wishes to see colors used to represent the number instead of bullet points to indicate status of an API.