-
Notifications
You must be signed in to change notification settings - Fork 108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add UI/frontend test to pbench-dashboard #2980
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
.vscode/ | ||
tmp/ | ||
wtlog.log | ||
geckodriver.log | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
ENV/ | ||
env.bak/ | ||
venv.bak/ | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
[WTRobot](https://pypi.org/project/wtrobot/) is a keyword driven UI/frontend testing framework. | ||
|
||
## Setup | ||
|
||
### Install | ||
|
||
```console | ||
|
||
> pip install wtrobot | ||
|
||
``` | ||
Comment on lines
+7
to
+11
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think the blank lines here affect the formatted result, and I don't think they make the source code easier to read; so, I recommend omitting them. Ditto lines 27-31. |
||
|
||
> NOTE | ||
|
||
- Selenium_drivers folders have your selenium webdrivers geckodrivers(for firefox) and chromedrivers(for chrome and chromium) | ||
|
||
- If script fails due to drivers issue, you need to find appropriate selenium webdriver according to your browser version | ||
|
||
- [firefox](https://github.com/mozilla/geckodriver/releases) & [chrome/chromium](https://chromedriver.chromium.org/downloads) | ||
|
||
- Unzip or untar the executable and place in selenium_drivers dir. | ||
|
||
Comment on lines
+13
to
+22
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Placing "NOTE" in a block quote is an interesting formatting choice, but placing the bulleted list under it outside the block quote is weird. I suggest either including the bullet list in the block quote or replacing the Ditto lines 33-36. |
||
## Executing Script | ||
|
||
- Write all your test cases into test.yaml and execute | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Put "test.yaml" in mono-space font (i.e., in back-ticks). |
||
|
||
```console | ||
|
||
> wtrobot | ||
|
||
``` | ||
|
||
> NOTE | ||
|
||
- If config(in `config.json`) is missing on initial run, tool will ask you for few configuration question and create `config.json` file. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is no There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the phrase, "ask you for few configuration question", should be "ask you a few configuration questions". |
||
- Make sure files which you mention as config(in `config.json`) should exist else will exit with error. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This text seems sort of obvious, so I would omit it. However, if you want to include it, perhaps phrasing like, "If any files referenced in |
||
|
||
## Syntax of test.yaml file | ||
|
||
- Write your WTRobot test cases in `test.yaml` files | ||
|
||
``` | ||
|
||
sequence: | ||
|
||
- testcase 1 | ||
|
||
- testcase 2 ... | ||
|
||
test: | ||
|
||
- testcase 1: | ||
|
||
- scenario: <your test senario desc> | ||
|
||
- step 1: | ||
|
||
name: <your step desc> | ||
|
||
action: goto | click | input | hover | scroll ... | ||
|
||
target: text | xpath | css path | ||
|
||
value: <data> | ||
|
||
- step 2: | ||
|
||
... | ||
|
||
- testcase 2: | ||
Comment on lines
+50
to
+70
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The hierarchy looks damaged, here -- the indentations are missing -- and I don't think the blank lines help. |
||
|
||
... | ||
|
||
``` | ||
|
||
[sample example](examples/test.yaml) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the sample example really at |
||
|
||
[detailed syntax](examples/syntax_docs.rst) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where is |
||
|
||
- Scenario and name are just detailed text description about your test case scenario and steps, they are useful for detailed logging | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The comma here is problematic. If you like it, then you should omit "they are". If you want to keep "they are", then you should use an emm-dash or semicolon instead of the comma...or, use a period and put "they are" in its own sentence. |
||
|
||
- There are only four important section to be considered while writing this script file | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. s/section/sections/ Also, I would omit "only". |
||
|
||
- `action`: what to perform (e.g. click, input and etc) | ||
|
||
- `target`: on what to perform (e.g. Text widget on web page, xpath of widget and etc) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The example you showed above There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should "Text" be down-cased? |
||
|
||
- `value`: with what data (e.g. if an input field then what value to input) | ||
|
||
- `assert`: after performing some action, what you want to validate. | ||
|
||
- It can be text, element, url |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
{ | ||
"script_filepath": "test.yaml", | ||
"browser": "firefox", | ||
"webdriver_path": "selenium_drivers/geckodriver", | ||
"locale": "en_US", | ||
"log": "wtlog.log", | ||
"dev": false | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Trailing newlines are generally a good idea; if nothing else, it keeps GitHub from showing the big red warning icon! |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
wtrobot |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
Driver version currently shipped: | ||
- Chrome/Chromium = v105.0.5195.19 | ||
- Firefox(gecko) = v0.31.0 | ||
|
||
> NOTE: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As I commented above, this is an "interesting" use of block quote, which I suggest replacing with bold and/or italics or a four/five-level header. |
||
- Selenium_drivers folders have your selenium webdrivers geckodrivers(for firefox) and chromedrivers(for chrome and chromium) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about
|
||
- If script fails due to drivers issue, you need to find appropriate selenium webdriver according to your browser version(choose latest if your browser is upto to date) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Are those version numbers browser versions or driver versions and how are they related? If they're driver versions, is there a key for relating those to browser versions? Finally, since browser independence is going to be a strong requirement, is there a way to have the test system pre-configured for both so that we can easily wire a functional test system to be sure both run correctly? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This statement seems self-evident, so I would either omit it or rephrase it along the lines of, "You must provide a Selenium webdriver which matches your choice of browser and browser version." |
||
- [firefox](https://github.com/mozilla/geckodriver/releases) & [chrome/chromium](https://chromedriver.chromium.org/downloads) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is no need to have these references in their own list of one bullet. Just append them to the previous line. |
||
- Unzip or untar the executable and place in selenium_drivers dir. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. s/in selenium_drivers dir/in the `selenium_drivers` directory/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
sequence: | ||
- testcase 1 | ||
- testcase 2 | ||
test: | ||
- testcase 1: | ||
- scenario: invalid pbench-dashboard login | ||
- step 1: | ||
action: goto | ||
target: http://10.1.170.201 | ||
Comment on lines
+4
to
+9
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This structure seems a little...odd. It makes reasonable sense that a A more natural structure might look like this:
That is, Although, if you really like the "name attribute as key", it might look like this:
where |
||
targets: | ||
- http://10.1.170.201 | ||
Comment on lines
+9
to
+11
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't want to hardcode IP addresses here; is there a way of importing these by variable from outside? This, in particular, is my personal test server and ideally nobody else would be depending on it at all ... in general, we'll be running our functional tests against a transient pod set up and torn down by the Jenkins pipeline and we need to run the dashboard tests against that server.
Comment on lines
+9
to
+11
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to specify both There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In all the steps the two keys always seem to have the same value.... |
||
- step 2: | ||
action: click | ||
target: Login | ||
targets: | ||
- Login | ||
- step 3: | ||
action: input | ||
target: //*[@id="username"] | ||
value: user | ||
targets: | ||
- //*[@id="username"] | ||
- step 4: | ||
action: input | ||
target: //*[@id="password"] | ||
value: password | ||
targets: | ||
- //*[@id="password"] | ||
- step 5: | ||
action: click | ||
target: Login | ||
wait after action: 3 | ||
assert: text=Bad login | ||
targets: | ||
- Login | ||
- testcase 2: | ||
- scenario: Valid pbench-dashboard login | ||
- step 1: | ||
action: goto | ||
target: http://10.1.170.201 | ||
targets: | ||
- http://10.1.170.201 | ||
- step 2: | ||
action: click | ||
target: Login | ||
targets: | ||
- Login | ||
- step 3: | ||
action: input | ||
target: //*[@id="username"] | ||
value: vishalvvr | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't want to hardcode user and password, either; and certainly not to your account! While we could stand up a CI pipeline pod with a pre-configured test user, it'd be much better to have this imported from outside via an environment variable or similar. |
||
targets: | ||
- //*[@id="username"] | ||
- step 4: | ||
action: input | ||
target: //*[@id="password"] | ||
value: Redhat@123 | ||
targets: | ||
- //*[@id="password"] | ||
- step 5: | ||
action: click | ||
target: Login | ||
targets: | ||
- Login | ||
assert: text=Logged in successfully! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an equivalent for Chrome?