hg: don't use custom diff arguments when generating diffstats for rss feed
With Mercurial 4.7, it is no longer possible to pass arguments like `git=True` to `ctx.diff()`.
This will change the reported data slightly, but the difference isn't important. Also, the code should be refactored to do something similar to what we do in other places.
522cfb2be9e1 introduced test_custom_hooks_preoutgoing, but thorough testing now reveals that the test failed with Mercurial 4.0 to 4.1.0 . Instead of investigating and fixing code or test, just accept this more than one year old version as minimum.
ishell: use traitlets.config and require ipython >= 4.0.0
Since version 4 was released in 2015, ishell has warned: IPython/config.py:13: ShimWarning: The `IPython.config` package has been deprecated. You should import from traitlets.config instead. "You should import from traitlets.config instead.", ShimWarning)
Goal is to reduce the gigantic 'setup' page in size, and make the information more understandable. Currently, the different topics do not really belong together. This is one small step towards that goal.
docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
The existing page on VCS support was a mix of information needed to setup Kallithea with respect to version control systems, with information regarding using version control systems (or specific aspects of it) with Kallithea.
Move the first part to the Administrator Guide, and rebrand the second part as VCS Usage Notes.
In vcs_notes.rst, the general info is moved above the Mercurial-specific part, but otherwise left untouched.
db: drop constraint that started failing with MariaDB 10.2 / MySQL 5.7 (Issue #324)
The constraint was to prevent simple recursive parent references, but it only checked direct parents. We thus have to rely on the high level application maintaining the invariant anyway.
Reorganize the index page of the manual into three main sections: administrator guide, user guide, developer guide. In principle, administrators, users and developers are independent roles that can be fulfilled by different people.
'Administrators' install, configure and maintain Kallithea. They have 'admin' privilege in the web interface.
'Users' do not have access to the Kallithea installation files. They can only access the web interface, and generally do not have 'admin' privilege. They may be the owner of repositories, repository groups, or user groups, and as such modify the settings of these objects, even though they cannot modify the settings of Kallithea itself.
'Developers' modify the Kallithea source code. They may or may not contribute these changes back to the Kallithea community.
The 'Readme' section is a bit outside of this classification. As it serves as a basic introduction to Kallithea, it is kept first.
The 'Index' and 'Search' functionality are not really 'part of' the manual, so should not be under 'Other topics'. Instead they are put at the very top so they are readily visible.
This commit is only changing the index page. Subsequent commits can move and split some documentation files in the repository to mirror this structure. For example, the 'Version control systems support' page has information for administrators (ini settings and importing existing repositories) as well as for users (handling Mercurial subrepositories).
utils: move repo_name_slug to utils2 to prevent import cycle on setup_db
After commit 57a733313e4f, 'gearbox setup-db -c my.ini' fails with an import cycle as follows:
Traceback (most recent call last): File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-release/bin/gearbox", line 11, in <module> sys.exit(main()) File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-release/lib/python2.7/site-packages/gearbox/main.py", line 199, in main return gearbox.run(args) File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-release/lib/python2.7/site-packages/gearbox/main.py", line 145, in run return self._run_subcommand(remainder) File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-release/lib/python2.7/site-packages/gearbox/main.py", line 149, in _run_subcommand subcommand = self.command_manager.find_command(argv) File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-release/lib/python2.7/site-packages/gearbox/commandmanager.py", line 78, in find_command cmd_factory = cmd_ep.resolve() File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-release/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2324, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "/home/tdescham/repo/contrib/kallithea/kallithea-release/kallithea/lib/paster_commands/setup_db.py", line 27, in <module> from kallithea.lib.db_manage import DbManage File "/home/tdescham/repo/contrib/kallithea/kallithea-release/kallithea/lib/db_manage.py", line 47, in <module> from kallithea.model.repo_group import RepoGroupModel File "/home/tdescham/repo/contrib/kallithea/kallithea-release/kallithea/model/repo_group.py", line 35, in <module> import kallithea.lib.utils File "/home/tdescham/repo/contrib/kallithea/kallithea-release/kallithea/lib/utils.py", line 48, in <module> from kallithea.model.repo_group import RepoGroupModel ImportError: cannot import name RepoGroupModel
i.e. kallithea.model.repo_group wants to import kallithea.lib.utils which in turn wants to import kallithea.model.repo_group.
In fact there exists kallithea.lib.utils and kallithea.lib.utils2. The current split is that 'utils2' contains 'simple' utilities, none of which depend on kallithea models, controllers, ... In contrast, 'utils' does rely on such kallithea classes.
As kallithea.model.repo_group was only include kallithea.lib.utils for its repo_name_slug method, which has no dependency on other kallithea classes, move that method (and its dependent recursive_replace) to kallithea.lib.utils2 instead. This fixes the import cycle.
pullrequests: don't show empty "additional changesets" (issue #280)
When opening a pullrequest on a revision range including the tipmost revision, and then pushing a new revision on top of that, the PR page shows: This pullrequest can be updated with changes on ...: and then nothing.
This is because the available revisions looped over are in 'avail_cs' but the guard around the loop is checking on 'avail_revs'. The former, while derived from avail_revs, can become empty under circumstances like this one.
Fix the problem by changing the guard checking avail_cs rather than avail_revs, and making sure the printed message is aligned to 'No additional changes found'.
repos: introduce low level slug check of repo and group names
The high level web forms already slug-ify repo and repo group names. It might thus not create the exact repo that was created, but the name will be "safe".
For API, we would rather have it fail than not doing exactly what was requested.
Thus, always verify at low level that the provided name wouldn't be modified by slugification. This makes sure the API provide allow the same actual names as the web UI.
This will only influence creation and renaming of repositories and repo groups. Existing repositories will continue working as before.
This is a slight API change, but it makes the system more stable and can prevent some security issues - especially XSS attacks.
repos: only allow api repo creation in existing groups
Fix problem with '../something' paths being allowed; '..' will always exist and can't be created.
This also introduce a small API change: Repository groups must now exist before repositories can be created. This makes the API more explicit and simpler.
repos: introduce low level slug check of repo and group names
The high level web forms already slug-ify repo and repo group names. It might thus not create the exact repo that was created, but the name will be "safe".
For API, we would rather have it fail than not doing exactly what was requested.
Thus, always verify at low level that the provided name wouldn't be modified by slugification. This makes sure the API provide allow the same actual names as the web UI.
This will only influence creation and renaming of repositories and repo groups. Existing repositories will continue working as before.
This is a slight API change, but it makes the system more stable and can prevent some security issues - especially XSS attacks.
repos: only allow api repo creation in existing groups
Fix problem with '../something' paths being allowed; '..' will always exist and can't be created.
This also introduce a small API change: Repository groups must now exist before repositories can be created. This makes the API more explicit and simpler.
Some hook names are quite long and are visually truncated in the admin section. Use the same principle as on the System Info: hide overflow and add ellipsis, use a title attribute to let users discover the full name on hover.
admin: system info: add title attribute for truncated items
On the System Info page, the item names may be truncated, e.g. "Kallithea configur...". In these cases, it is not possible for a user to retrieve the full name.
Add a title attribute that will show the full name on hover.
admin: hooks: only flash 'Updated hooks' if there are changes
Clicking 'Save' on the hook administration page currently always renders the flash message 'Updated hooks' even if nothing was changed. This may be particularly confusing when the action you intended to do got an error, e.g. adding a hook that already exists, adding a builtin hook, ...
Instead, compare the old and new value when editing a hook, and only save and create the flash if they are different.
For this to be work correctly in test, the old value needs to be passed as well like in the real situation, otherwise the 'zip' operation will return an empty list.
admin: hooks: prevent editing of builtin hooks (issue #226)
Builtin hooks are supposed to be read-only, but it was still possible to 'add' a new hook with the same name as an existing built-in one, changing its value.
admin: hooks: restore delete functionality as intended
Commit 9d34bea3059d9abd0d912f37a2475ee67c8e2918 ("style: various minor-ish markup changes, preparing for Bootstrap") partially broke the delete functionality of hooks. When clicking the delete button, the hook is deleted via AJAX and then the corresponding form group is intended to be removed visually. This relies on an 'id' attribute on the form-group div.
The mentioned commit moved the div with the 'id' attribute outside the loop iterating over the different hooks, so that there no longer is a div with the id expected by the delete button. The hook would still be deleted, but the page visually still looks the same until refresh/Save.
Move the diff back into the loop. This causes a little more visual separation between the different hooks, but still looks OK. The layout of the built-in hooks (that can't be removed) is left untouched. Making the same change there would be possible but there the extra vertical whitespace seems somewhat unnecessary, although this is personal.
tests: fix caching issue in test_ip_restriction_git
Following test failure is observed in TestVCSOperations.test_ip_restriction_git:
―――――――――――――― TestVCSOperations.test_ip_restriction_git ―――――――――――――――――― kallithea/tests/other/test_vcs_operations.py:584: in test_ip_restriction_git assert re.search(r'\b403\b', stderr) E assert None E + where None = <function search at 0x7fb9772da578>('\\b403\\b', "Cloning into '/tmp/kallithea-test-SZhXDz/vcs_operations-krPNvZ'...\n") E + where <function search at 0x7fb9772da578> = re.search ------------------------- Captured stdout call ---------------------------- *** CMD git clone http://test_admin:test12@127.0.0.1:45291/vcs_test_git /tmp/kallithea-test-SZhXDz/vcs_operations-krPNvZ *** stderr: "Cloning into '/tmp/kallithea-test-SZhXDz/vcs_operations-krPNvZ'...\n"
The test is setting up IP restrictions, verifying that access is no longer possible, then clears the restriction. There already were sleeps after clearing the restrictions, in order for the cache to expire and have the setting take effect. But there was no sleep on the enabling of the IP restriction, allowing situations where the code would still run without restriction, and thus allow the access, failing the test.
The failure has only been observed on test_ip_restriction_git, but the change is also made for test_ip_restriction_hg.
The existing sleeps after restriction clearing are moved up to the 'finally' clause to make it clear to which code they belong.
hg: fix continuation binding of callback variables
The get_file_annotate generator yields a callback. That callback happened to look at the current state of the generator. It happened to work anyway, apparently because the callback always was called immediately, before the generator state changed. But for example if the get_file_annotate was buffered into a list, the callback would return the same result each time.
Instead, bind the generator iterator variables as named parameter values for the callback function. That will make sure the callback always returns the right value.
If git user email is not set yet, git requires the EMAIL environment variable to be set. If it wasn't set in the test environment, test_add_submodule would fail with something like:
style: use monospace on all multi-line form inputs
Multi-line form input (textarea) is used for PR comments, PR descriptions, repository descriptions and 'HTML/CSS/Javascript customization block'. In the first three cases, the rendered result is displayed in monospace, so the corresponding textarea should be as well. As the expected content of the HTML/CSS/Javascript block is code, it makes perfect sense to use monospace there too.
The 'formatted-fixed' class is applied to PR comments, PR descriptions and repository descriptions. It already preserves whitespace via 'whitespace: pre-wrap', but the font is not monospace. As a result, diagrams or other ASCII art do not display well.
The ini setting 'initial_repo_scan' caused a repository scan on each startup of Kallithea. The accompanying comment already warns that the feature should be disabled after the first run to improve startup time.
Now that setup-db is performing the initial repository scan, and considering that administrators can always request a new scan using 'gearbox repo-scan' or via the web interface, the 'initial_repo_scan' feature is no longer considered useful and is removed with this commit.
setup-db: perform an initial repository scan as stated by the docs (Issue #302)
The documentation, as well as the prompt text inside setup-db itself, states that the repository root location will be scanned automatically for existing repositories. However, this is not actually the case. Only exception is when the 'initial_repo_scan' is set to True in the ini file; in that case the scan is done on each start of Kallithea.
Add the required repo scan logic at the end of setup-db, after setting up the application completely (the database has only just been set up).
The app setup code from BasePasterCommand is duplicated - this command doesn't fit in and need to run both before and after database creation.
The scan call from kallithea/config/app_cfg.py is also duplicated - that will be removed next.
admin: auth: make sure list of auth modules is consistent
When authentication modules are enabled, but fail to be enabled e.g. due to missing dependencies (pam, ldap), the list of enabled plugins still contains the failing module. However, the 'Enabled/Disabled' button correctly shows Disabled, causing a mismatch between both.
Worse, the mismatch cannot be corrected by clicking the Enabled/Disabled button, one needs to manually clear the problematic entry in the list of enabled plugins.
Fix by always populating the list with the actually enabled plugins, not those requested by the user in case there are failures.
If 'gearbox repo-scan' does not add nor remove anything, the output is:
Now scanning root location for new repos ... Scan completed. Added: - Missing: -
These empty lists of results are not very helpful. Instead, rework the code so that a list of added/removed/missing repositories is only printed if it actually contains something.
style: use @panel-default-border for vertical line number / diff separator lines
These lines appear right inside panels and have the same role as border lines. They will thus generally be styled with the same color and should use the same variable.
repos: clean up table button markup to give proper spacing
Before, it used multiple divs with pull-left to make plain buttons and buttons within forms render horizontally. But by doing so, they were rendered without any space between the buttons.
Instead, style forms after buttons to be shown inline. The whitespace will thus give the necessary spacing.
less: avoid PR reviewer list being right-aligned on wide screens
#pr-summary had max width md, but the reviewer list was in the last column of the grid with the full page width.
The pull request summary should not be too wide, so the reviewers are not to far away. But to achieve this the whole .pr-box needs the max-width not just #pr-summary.
less: generate map files when building style.css with 'npm run less'
The map files allow web development tools ("Firebug" style) to show the .less and .css source instead of the generated style.css . The generated style.css will get a trailing comment with a reference to kallithea/public/css/style.css.map . It will not have any impact on runtime.
1) Not using 'async defer' - it's not used anywhere else, plus I have no idea of what exactly it does.
2) The recaptcha div has an ID so the label for="recaptcha_field" can reference it (although I'm not certain a div can technically have a label, but it's better than removing the label, or the for="").
graph: make sure graph is shown next to the table it belongs to
For some screens sizes (for example around 800 px), the graph with changes available for PR update wasn't shown next to the corresponding table. The two boxes would float vertically and be rendered after each other.
To fix that, use the same trick as before 02aef1484695: set the graph height to 0 and give the table a left margin leaving room for the graph.
(The fancyness of Bootstrap grids does in this case seem to only get in the way. Plain percent would perhaps work better.)