Commit Graph

214 Commits

Author SHA1 Message Date
Greg Price e8bf7d23bf provision: Remove transitional Python 2->3 logic in setup_venvs.
At this point if we were accidentally using `/srv/zulip-venv` for
anything, we'd have run into it by now.  So just drop the bit of
historical logic that we had to ensure that.
2017-12-18 21:59:51 -08:00
Greg Price 137c0e65bb tools: Revert to Python 2 typing syntax for now.
This reverts commit 66261f1cc.  See parent commit for reason; here,
provision worked but `tools/run-dev.py` would give errors.

We need to figure out a test that reproduces these issues, then make a
version of these changes that keeps that test working, before we
re-merge them.
2017-12-13 10:38:15 -08:00
rht 66261f1cc3 tools: Use Python 3 syntax for typing in many files. 2017-12-12 17:42:57 -08:00
rht bff736868e Generate custom-icon-webfont on each provision or update-prod-static.
Fixes #7354.
2017-11-20 16:36:49 -08:00
Greg Price 13aa23eb40 provision: Run clean-unused-caches.
This prevents the caches in /srv from growing to fill up the disk --
e.g., on my laptop after 6 months of regular development the venv cache
was 12G and the NPM cache 5G, making them by far the largest disk hogs
on the machine.

It costs about 0.4s, apart from any time spent actually removing
things.  This is a little annoyingly slow to be adding to every
provision, and seems like it could be optimized, but I think already
worth it as is.
2017-11-18 15:04:08 -08:00
Tim Abbott 638eb7a8e4 docs: Update links to ReadTheDocs to always use https.
This is better security practice.

We also add a lint rule to enforce this for the future.
2017-11-16 10:59:24 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Tim Abbott 774380dcb8 capitalization: Allow enabled/disabled to be lower-case. 2017-11-15 17:39:09 -08:00
rht 72ce1c6501 Use a more concise `mkdir -p` when the situation permits. 2017-11-14 09:47:09 -08:00
Steve Howell ba51078418 Simplify CSS linter and clean up CSS.
The CSS linter was pretty hard to reason about.  It was
pretty flexible about certain things, but then it would
prevent seemingly innocuous code from getting checked in.

This commit overhauls the pretty-printer to be more composable,
where every object in the AST knows how to render itself.  It
also cleans up a little bit of the pre_fluff/post_fluff logic
in the parser itself, so comments are more likely to be "attached"
to the AST node that make sense.

The linter is actually a bit more finicky about newlines, but
this is mostly a good thing, as most of the variations before
this commit were pretty arbitrary.
2017-11-13 12:43:43 -08:00
rht 7a96b8c7ad tools/lib: Text-wrap long lines exceeding 110. 2017-11-10 16:24:09 -08:00
rht e3eebf3be0 Remove inheritance from object. 2017-11-06 08:53:48 -08:00
rht 691598a88b py3: Remove "from six.moves import range".
This is no longer required, since in Python 3, this is what the range
built-in does.
2017-10-17 23:28:14 -07:00
rht 9c5e0971f4 sanity_check: Check a few packages.
This just makes our check for whether we're actually in a Zulip
development environment better.
2017-10-17 22:21:33 -07:00
Tim Abbott 508686bed4 provision: Fix buggy bash_profile management code.
Apparently, the previous version would just append to .bash_profile
every single time :(.
2017-10-17 21:22:32 -07:00
Tim Abbott 3fed995a41 provision_version: Move to under UUID_VAR_PATH. 2017-10-17 21:15:58 -07:00
Tim Abbott 3983ebce9a test_fixtures: Move test_db_status state under UUID_VAR_PATH.
This should mean that maintaining two Zulip development environments
using the same Git checkout no longer has caching problems keeping
track of the migration status.
2017-10-17 21:15:58 -07:00
Tim Abbott a7c639cff7 provision: Use UUID in path to compilemessages hash. 2017-10-17 21:15:58 -07:00
Tim Abbott 2ae2a94444 provision: Stop using shared var/ for caching apt state.
This didn't work at all when one did a `vagrant destroy` and then
`vagrant up`, because the cache state would be preserved even though
the machine is gone.

Fixes #5981.
2017-10-17 21:15:58 -07:00
derAnfaenger df948c7ea5 Remove useless shebang lines. 2017-10-11 12:52:36 -07:00
rht 34bafa654c mypy: Fix future syntax errors and other minor mistakes.
When we move to the Python 3 mypy syntax, we can't reference a class
before its definition.
2017-10-10 22:27:16 -07:00
rht dcd80e6650 travis/setup-backend: Remove the '--travis' flag in tools/provision.
Whether the env is Travis or not, this has been detected via $TRAVIS env var.
Proof:
6fbf41bdbc/tools/provision (L38)
2017-10-06 11:20:25 -07:00
rht 2c2adbf647 tools/lib: Remove unused imports. 2017-10-02 10:13:46 -07:00
rht f15bdce90e tools: Remove print_function.
Tweaked by tabbott to exclude the linter libraries.
2017-09-29 15:44:56 -07:00
Rhea Parekh 0e5fdcea3f provision: Run apt-get update on catching an error.
For broken apt cache, we run the command apt-get update on catching
the error.
2017-09-29 15:38:23 -07:00
rht bf4eda7374 tools: Remove absolute_import in most tools.
Tweaked by tabbott to not remove it from lister.py, linter_lib, and
friends, since those are intended to support both Python 2 and 3
(we're planning to extract them from the repository).
2017-09-29 12:28:43 -07:00
Eeshan Garg bb34ba37b5 tools: Rename build_pygments_data.py -> build_pygments_data.
Dropping the file extension (.py) conforms to our naming
convention for such scripts.
2017-09-26 16:00:41 -07:00
Umair Khan 5d00f29c68 capitalization: Ignore LDAP. 2017-09-26 15:58:17 -07:00
Tim Abbott 8e2ae66fe6 provision: Update apt operations text. 2017-09-25 14:47:18 -07:00
Tim Abbott 16ed23414d provision: Fix spelling of apt_dependencies_hash.
Also use a variable to avoid duplicating the path.
2017-09-25 14:46:46 -07:00
Eeshan Garg 390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Vishnu Ks 61e48734a7 provision: Add moreutils in APT common dependencies.
This fixes an issue where `tools/update-locked-requirements` would fail.
2017-09-22 14:04:57 -07:00
Vishnu Ks f4c69bb126 tools: Create tool for inlining email CSS during provision. 2017-09-16 07:17:48 -07:00
Tim Abbott 126171cb76 api: Remove root_domain_uri from API page links.
These pages should be references the current server's UI anyway.
2017-08-28 14:13:37 -07:00
Tim Abbott a0a1fe1512 settings: Rename SERVER_URI to ROOT_DOMAIN_URI.
This should be a lot less confusing.

See #6013 for discussion.
2017-08-28 14:09:28 -07:00
Tim Abbott 90012f8109 find_add_class: Don't check frontend tests. 2017-08-27 14:04:46 -07:00
Tim Abbott ce1a8bf5b2 find_add_class: Include full path names in error messages. 2017-08-27 14:04:46 -07:00
rht 437b2a3146 Update "MacOS" text to "macOS" 2017-08-26 09:00:42 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Greg Price b7ec1b55dd py3: Cut Python 2 codepaths from provision. 2017-08-16 17:54:43 -07:00
Anirudh Jain 28944b6c94 provision: Partially add zesty to supported systems.
We can't fully support it until we fix the tsearch_extras availability
issue, but for now, this is an improvement.

Tweaked by tabbott to cover the outstanding tsearch_extras issue.
2017-08-15 21:51:19 -07:00
neiljp (Neil Pilgrim) 1a3d8ced78 mypy: Allow Optional parameters in html_branches.py. 2017-08-15 17:50:18 -07:00
Greg Price d3a69da829 py3: Fix or remove (almost) all references to Python 2 venv in docs.
I think soon we'll put the Python 3 venv at `/srv/zulip-venv` and
make `/srv/zulip-py3-venv` just an alias to that (then remove the
alias too), but for now the old name is helpful for spotting places
that need an update.

The `/srv/zulip-venv` name still appears in codepaths used by Travis
tests.
2017-08-08 15:48:12 -07:00
neiljp (Neil Pilgrim) b782db48e1 mypy: Remove superfluous older 'type: ignore' annotations. 2017-08-08 11:27:51 -07:00
neiljp (Neil Pilgrim) 982c0481d0 mypy: Mark log_file as Optional[str] in test_server.py. 2017-08-07 21:27:50 -07:00
Pweaver (Paul Weaver) 1afaa67c7c deps: Change npm to yarn for reliablity, security, and speed. 2017-08-05 12:29:06 -07:00
rht 532bd6eaa6 Ensure virtualenv is run throughout the entire Travis dev provision. 2017-08-01 15:29:13 -07:00
Jack Zhang 5a8b1e6253 hotspots: Implement hotspots frontend. 2017-07-28 16:34:15 -07:00
Greg Price 7cd621bc9d webhooks/updown: Fix a JsonableError to provide a real error message. 2017-07-24 16:41:22 -07:00
Tim Abbott 5d05cc7294 scripts: Rename NPM_CACHE_PATH to NODE_MODULES_CACHE_PATH. 2017-07-21 17:28:20 -07:00
Jack Zhang e3ef056991 portico: Update registration form help messages. 2017-07-21 13:09:06 -07:00
Aditya Bansal 7840ae0fe4 template_parser: Add 'with' to Django/Jinja block elements.
Fixes #5863.
2017-07-19 16:39:54 -04:00
Tim Abbott 680381c9d0 setup_venv: Add virtualenv as a dependency for Debian.
Apparently, this was missing on Xenial, and we just never noticed
because the package was installed by default on Ubuntu.

It doesn't exist yet in Trusty.
2017-07-14 17:22:20 -07:00
Tim Abbott 4906a47c1a provision: Disable tsearch_extras for Debian. 2017-07-14 17:22:20 -07:00
Tim Abbott 9efc017e8c provision: Add postgres versions for Debian stretch. 2017-07-14 17:22:20 -07:00
Yago González 2079d84018 lint: Add mobile stores to the capitalization exceptions. 2017-07-14 16:27:17 -07:00
Harshit Bansal ebdd8c9e3d emoji: Switch to using `emoji-datasource` npm package.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
  `emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
  package.

Bumps PROVISION_VERSION.

Fixes: #4730.
2017-07-13 21:52:01 -07:00
Harshit Bansal 420327b558 provision: Install NPM packages in the beginning of provisioning process.
NPM packages should be installed at the beginning of the provisioning
process so that later in the provisioning process if a script requires
any NPM package it can use it. Earlier, we were installing NPM packages
in the last as the installation process can fail due to network issues
but since we now retry in case the installation fails, they can be
installed safely at the beginning of the process as well just like apt
packages.
2017-07-13 21:52:01 -07:00
Tim Abbott f2379d66a2 i18n: Fix capitalization of Zulip Botserver download code. 2017-07-07 17:02:39 -07:00
Tim Abbott 919c610abf capitalization: Add APNS to list of valid acronyms. 2017-07-07 10:43:02 -07:00
Tim Abbott 774216713b provision: Document more clearly the is_production_travis block. 2017-07-06 17:39:49 -07:00
sinwar 49db0855d2 provision: Avoid spending ~3s on RabbitMQ in no-op case.
Tweaked by tabbott to reorganize the logic more clearly.

Fixes #5180.
2017-07-06 17:38:39 -07:00
sinwar 3d0b5b14be provision: Avoid spending ~1.5s on compilemessages in no-op case.
This brings the total no-op provision time down to around 7.3s!

Tweaked by tabbott to clean up the code, close file descriptions, and
avoid issues with a somehow corrupted file.

Fixes #5185.
2017-07-01 11:27:54 -07:00
Yago González 7e3154f7bc lint: Ignore tildes in mustache tag template checker.
Handlebars allows putting tildes (~) by the braces in mustache tags to
strip whitespaces on that side of the tag.

    {{#if foo~}}
        <p>Bar</p>
    {{~/if}}

This way, the linter ignores any potential tilde at the ends of Handlebars
tags so we can use this feature without lint fails.
2017-06-26 19:33:25 -04:00
sinwar 0e24930770 provision: Avoid spending ~3s on apt operations in no-op provision.
Fixes #5186
2017-06-22 09:37:40 -04:00
Tommy Ip e9fd41bb66 lint: Check if the first word of a commit message is imperative.
Only check the first word using a black list approach to reduce
force positive as shown in the current implementation.
2017-06-19 07:01:52 -04:00
Aditya Bansal a2b5d6efcb provision: Add wrapper to retry apt-get install if it fails first time. 2017-06-14 21:49:15 -07:00
Aditya Bansal d2f0e17454 linter: Fix bug resulting in <pre> being indented cyclically. 2017-06-13 08:39:59 -04:00
Aditya Bansal a9e0dbe085 pep8: Add compliance with rule E261 pretty_print.py. 2017-06-04 15:05:51 +05:30
Tim Abbott cecf7f1740 provision: Don't regenerate dev database unnecessarily.
This reuses the work we did some time ago to avoid regenerating the
test database unnecessarily.

In addition to being a nice convenience for developers (since any
accomulated test data is still available), this also saves about half
the time consumed in a no-op provision.

Fixes #5182.
2017-06-02 13:27:34 -07:00
Steve Howell 5f48b51b60 provision: Add `cd /srv/zulip` to profile.
I didn't bump the provision version on this change, as it
won't break anything if somebody fails to make this change.
2017-05-29 08:40:49 -07:00
Tim Abbott 9b8641d9eb lint: Add iOS to list of valid weird capitalizations. 2017-05-26 20:59:32 -07:00
Umair Khan ad5780c03e lint: Mark alt argument for translation. 2017-05-26 15:28:46 -07:00
rht a1f82e02d6 python: Replace os.system with subprocess.call.
Generally, we avoid os.system, since it shells out and thus can be
a cause of security issues.
2017-05-26 15:03:16 -07:00
Tim Abbott 8aaf9b1426 Revert "Fix emoji cache setup when not using provision.py."
This reverts commit ab2cfadcaa.

This broke production installation.
2017-05-24 10:35:41 -07:00
Eklavya Sharma 7636972a6c mypy: Wrap return value in Optional. 2017-05-23 21:56:50 -07:00
Eklavya Sharma cc1937c8d5 mypy: Use Optional with strings where required. 2017-05-23 21:56:50 -07:00
Jordan Gedney a9f4d56a95 mypy: Add type to info dictionary. 2017-05-22 11:19:06 -07:00
Carey Metcalfe ab2cfadcaa Fix emoji cache setup when not using provision.py.
Moves creating the emoji folder from the provisioning script to
the build_emoji script.

Fixes the fact that the emoji cache directory wasn't being created
when not using the provision.py script.
2017-05-22 11:08:43 -07:00
Aditya Bansal 9fb6d976ca pep8: Add compliance with rule E261 to test_server.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 46b3c14876 pep8: Add compliance with rule E261 to html_grep.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 47e7437932 pep8: Add compliance with rule E261 to html_branches.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal ffb8440eef pep8: Add compliance with rule E261 to graph.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal c0e1b49b68 pep8: Add compliance with rule E261 to find_add_class.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal fef732d85b pep8: Add compliance with rule E261 to css_parser.py. 2017-05-07 23:21:50 -07:00
Brock Whittaker c8902d5d71 settings: Make settings overlay more responsive.
This doesn't completely fix settings responsiveness, but it's a big
step along the way.  Outstanding issues include:

1. When switching tabs from settings to organization, it will launch
the first item which is more annoying in this view since it brings you
into that tab. Haven’t decided on an elegant solution to this yet.

2. Sidebar scrolling doesn’t work. I have to restructure how the top
section and bottom sections of content are displayed to fix this.
Likely by enforcing min-height of 100% - bottom height on the top piece.

3. Most of it is actually reasonably responsive but some isn’t, and
should be fixed on a case-by-case.
2017-05-05 13:42:09 -07:00
Tommy Ip 19adf2e327 requirements: Upgrade gitlint to 0.8.2.
As of Gitlint v0.8.2, the --extra-path option now accepts both
directories and standalone python modules.
2017-05-05 09:59:39 -07:00
Steve Howell e3edc4d829 Send welcome messages for new streams. 2017-05-01 16:23:38 -07:00
Cory Lynch 0965c43238 Add typeahead for syntax highlighting languages.
Modified composebox_typeahead.js to recognize the triple backtick
and tilde for code blocks, and added appropriate typeahead functions
in that file and in typeahead_helper.js.

Additionally, a new file pygments_data.js contains a dictionary of
the supported languages, mapping to relative popularity
rankings. These rankings determine the order of sort of the
languages in the typeahead.

This JavaScript file is actually in static/generated/pygments_data.js, as it
is generated by a Python script, tools/build_pymgents_data.py. This is
so that if Pygments adds support for new languages, the JavaScript file
will be updated appropriately. This python script uses a set of popularity
rankings defined in lang.json.

Corresponding unit tests were also added.

Fixes #4111.
2017-04-28 17:22:59 -07:00
Neeraj Wahi 90a154e451 Add mobile auth redirect to custom URI scheme (zulip://).
This makes it possible for the Zulip mobile apps to use the normal web
authentication/Oauth flows, so that they can support GitHub, Google,
and other authentication methods we support on the backend, without
needing to write significant custom mobile-app-side code for each
authentication backend.

This PR only provides support for Google auth; a bit more refactoring
would be needed to support this for the GitHub/Social backends.

Modified by tabbott to use the mobile_auth_otp library to protect the
API key.
2017-04-28 11:47:35 -07:00
Steve Howell 7326971380 Extract stream_edit.js.
This code makes the right pane work in "Manage Streams" when
you are editing a stream subscription.  It handles basic
functionality (submitting forms, etc.), live updates, and
showing the pane as needed.

Most of the code here was simply moved from subs.js, but some
functions were pulled out of larger functions:

    live update:
        add_me_to_member_list
        update_stream_name
        update_stream_description

    collapse/show:
        collapse
        show_sub

We also now export subs.show_subs_pane.

We eventually want stream_edit not to call into subs.js, and
this should be fairly easy--we just need to move some shared
methods to a new module.
2017-04-25 09:57:32 -07:00
Tim Abbott 145fa0e5b2 capitalization: Flag Emoji One as allowed. 2017-04-24 23:15:39 -07:00
Tommy Ip 6ec32b37c3 lint: Fix lint rule error due to unordered list.
Tweaked to automatically sort by tabbott.
2017-04-22 11:38:36 -07:00
Tommy Ip 680e39ff55 linter: Add custom gitlint rule to check for imperative mood. 2017-04-21 15:43:19 -07:00
Joshua Pan 64984f5bf4 Rename tools/lint-all to tools/lint.
Fixes #4574.
2017-04-21 14:57:47 -07:00
Harshit Bansal 4470a3947b provision: Download emoji-data during provisioning.
Add code to download iamcal's emoji_pretty.json
(emoji-data/emoji.json) during provisioning.
2017-04-18 12:46:52 -07:00
Yago González e710110a9e i18n: Automatically strip Handlebars strings.
Some Handlebars strings contained whitespaces characters at their ends.
With this, such characters are removed, as well as multiple spaces
(like the ones produced by code indentation).

This also includes a couple of fixes that removes spaces that were
intentionally placed before/after the string to translate.
2017-04-14 17:37:25 -07:00
sinwar 4296d608ab test_server: Improve warning in venv check.
This (1) changes test_server to use the common `check_venv` method and
(2) improves check_venv to provide a clearer error message in the case
that you're inside Vagrant but not in a venv.

Tweaked by tabbott to borrow logic from run_dev.py.
2017-04-05 16:51:23 -07:00
Brock Whittaker 34f9ccb87c alerts: Change sidebar alerts to be at top of the screen.
This changes the alerts to be individual boxes that slide down from
the top of the screen for a better UI experience.
2017-04-03 16:22:05 -07:00
Steve Howell 2d52463b61 Extract ui_report.js.
This moves the implementations of error/report/message from
ui.js to ui_report.js.  They had been shimmed before, so calling
modules still use the same names to call the functions, but we
no longer need the shims.
2017-04-03 07:13:25 -07:00