Commit Graph

3287 Commits

Author SHA1 Message Date
Vishnu Ks 0fd6ff722b billing: Migrate /upgrade endpoint to JSON.
The fixture changes are because self.upgrade formerly used to cause a page load
of /billing, which in turn calls Customer.retrieve.

If we ran the full test suite with GENERATE_STRIPE_FIXTURES=True, we would
likely see several more Customer.retrieve.N.json's being deleted. But
keeping them there for now to keep the diff small.
2018-12-13 17:01:12 -08:00
Anders Kaseorg 149132348f lint: Update ShellCheck to 0.6.0.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-12 08:24:01 -08:00
rht a0b0ccf972 provision: Do not require lsb_release for centos. 2018-12-11 13:08:27 -08:00
rht c9d54f7854 puppet: Remove vendored puppetlabs apt and stdlibs dependencies.
This commit works by vendoring the couple functions we still use from
puppetlabs stdlib (join and range), but removing the rest of the
puppetlabs codebase, and of course cleaning up our linter rules in the
process.

Fixes #7423.
2018-12-11 13:03:26 -08:00
Vishnu Ks a86faecff5 tools: Rename tools/travis directory to tools/ci. 2018-12-10 11:07:26 -08:00
konraddeka e6be407e34 provision: Avoid spending time on update-authors-json.
This optimizes tools/provision by not running
`tools/update-authors-json --use-fixture` unless either the script
itself or its fixtures file (zerver/tests/fixtures/authors.json) was
changed.

Fixes #10991.
2018-12-09 22:21:51 -08:00
rht 9bfea2adba provision.py: Translate COMMON_APT_DEPENDENCIES to yum. 2018-12-09 21:54:02 -08:00
Sumanth V Rao 7d12e2019d provision: Generate static bots files using a function call.
This shaves about 350ms from the no-op provision runtime.

Fixes #10990.
2018-12-09 21:44:45 -08:00
Sumanth V Rao cae718f184 provision: Rename generate_zulip_bots_static_files to .py.
This should make it possible to call this script as a function without
starting up a new Python interpreter.
2018-12-09 21:42:40 -08:00
Tim Abbott 9e48b2f17f provision: Update comment for why we need puppet in development.
This should help avoid confusion like #10972.
2018-12-06 11:13:04 -08:00
Tim Abbott 9de1bd44e2 compatibility: Clean up linter logic for too-old client. 2018-12-05 15:04:21 -08:00
Greg Price fb7bfbe9ab compatibility: Require version 16.2.96+ for our Android app.
This release is from 2018-08-22, a little over 100 days ago.

It was the first release with the important fix so that when the
server advises it to stop displaying a notification because the user
has read the message (as the SEND_REMOVE_PUSH_NOTIFICATIONS server
setting enables), the app doesn't instead replace the notification
with a broken one reading "null".  We have that setting running now
on chat.zulip.org, and intend to roll it out more broadly soon.

The `# take 0` thing is a slightly absurd workaround for the fact
that our funky out-of-line way of marking lines to ignore doesn't
work right if there are multiple such lines in a given file that
are equal modulo leading and trailing whitespace.
2018-12-05 14:58:13 -08:00
Tim Abbott 6689913f77 ci: Enable test-documentation --skip-external-links in CI.
Fixes #10942.
2018-12-05 12:39:59 -08:00
Sumanth V Rao e1f33e0f6b test-documentation: Add --skip-external-links option.
This option causes test-documentation to only verify internal links
that we control and are important to be correct.  This prevents
test-documentation from flaking in CI due to issues with the dozens of
third-party blocks that we link to from various parts of our
documentation.

Tweaked by tabbott for comment clarity, and to also include
github.com/zulip links.

Fixes #10942.
2018-12-05 12:39:59 -08:00
Tim Abbott f5f320d5fc mypy: Print out link to documentation in error output.
This should help make details about how mypy works discoverable for
developers who are new to it.
2018-12-05 11:23:55 -08:00
Tim Abbott f694d01a1b test-backend: Expand desired files with full test coverage.
This should help catch a few classes of files that we weren't
correctly considering for potentially having full test coverage.
2018-12-05 11:01:55 -08:00
Steve Howell 67fba69b0c hashchange: Go to home page for bogus narrows.
This was the original intent of the code, and I think
it's the right behavior.
2018-12-04 17:16:38 -08:00
Steve Howell bcb142e68e hashchange: Move parse_narrow to hash_util.js.
The goal here was to enforce 100% coverage on
parse_narrow, but the code has an unreachable line
and is overly tolerant of bogus urls.  This will
be fixed in the next commit.
2018-12-04 17:16:32 -08:00
Anders Kaseorg 9ba860b4f2 Avoid double redirects to /login and then to /login/.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-04 16:12:58 -08:00
Tim Abbott 86536059b4 tools: Fix linter tests after SCSS rename.
This was introduced in 815388b023.
2018-12-04 09:43:56 -08:00
Vishnu Ks 788b98d041 portico: Add page for redirecting to a realm subdomain. 2018-12-04 09:35:35 -08:00
Tim Abbott 815388b023 lint: Fix CSS linter rules not running.
Apparently, our CSS linter rules haven't been running since we
migrated to SCSS.
2018-12-04 09:35:34 -08:00
Vishnu Ks 6de6b0ed31 emails: Extract CSS to email.css in digest.html. 2018-12-03 17:43:53 -08:00
Tim Abbott 07adda84da Revert "ci: Re-enable testing documentation links."
This reverts commit d8e5e2d960.

See https://github.com/zulip/zulip/issues/10942 for a plan for making
this something we can re-enable.
2018-12-02 18:34:35 -08:00
Tim Abbott 1303f8df20 docs: Fix missing quotes in `su zulip -c` documentation.
This fixes an actual user-facing issue in our mobile push
notifications documentation (where we were incorrectly failing to
quote the argument to `./manage.py register_server` making it not
work), as well as preventing future similar issues from occurring
again via a linter rule.
2018-11-30 12:12:27 -08:00
Tim Abbott a3b38d8402 docs: Document nginx configuration for development SSL.
This is a useful technique for developing OAuth integrations with Zulip.
2018-11-29 20:45:02 -08:00
Steve Howell ba243416e3 stream_color: Make stream color assignment more efficient.
We now let color_data keep its own state for
unused_colors, so that we longer have to pass in
a large list of unused_colors every time we want
to assign a new stream color.

This mostly matters at startup, where we might
be cycling through 5000 streams.  We claim all
the unused colors up front.

Each operation now has an upper bound of expensiveness,
where the worst case scenario is basically popping
off the first element of a list of <= 24 colors.

The algorithm is now deterministic, too, to make
it easier to test.  It's unclear whether random color
assignment ever had much benefit, and it made unit
testing the algorithm difficult.  Now we have 100%
line coverage.

Fixes part of #10902.
2018-11-29 13:00:25 -08:00
Anders Kaseorg 0bfcd2294e tools/update-prod-static: Avoid shelling out for mkdir, cp, mv, rm.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Anders Kaseorg 19bdf54f33 tools/setup/emoji/build_emoji: Avoid shelling out for touch, rm, mkdir, cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Tim Abbott d8e5e2d960 ci: Re-enable testing documentation links.
We've had a few unpleasant bugs with real documentation links being
broken, so we're going to make this more aggressive for now.

I think we instead want a more subtle option for suppressing failures
in some places but not others.
2018-11-28 09:52:04 -08:00
Tim Abbott 723fb60476 docs: Fix up some broken links output spam. 2018-11-28 09:49:39 -08:00
Rishi Gupta dd0126ff1b settings: Improve error message when deactivating the last user.
This PR was originally started by Rishi Gupta (see #10383).
2018-11-27 12:49:43 -08:00
Tim Abbott adf27aae4c python: Remove now-unnecessary str_utils library.
This library was absolutely essential as part of our Python 2->3
migration process, but all of its calls should be either no-ops or
encode/decode operations.

Note also that the library has been wrong since the incorrect
refactoring in 1f9244e060.

Fixes #10807.
2018-11-27 11:57:54 -08:00
Tim Abbott 58a6dafa68 lint: Exclude outbox from linter checks on "subject". 2018-11-27 11:05:25 -08:00
Tim Abbott d5f53164b4 docs: Suppress markdown line length linter errors. 2018-11-18 18:56:44 -08:00
Steve Howell eb4d279bbc Extract get_turtle_message().
This seems like kind of a silly function to extract
to topic.py, but it will theoretically help us sweep
"subject" if we change the DB.

It had test coverage.
2018-11-14 23:24:06 -08:00
Steve Howell cc33e4cd0c digest: Eliminate unneeded queries for hot convos.
We can easily keep track of messages by bucket from the
original loop through messages.
2018-11-14 23:24:06 -08:00
Steve Howell 5a539d44b0 subject -> topic: Fix notification tests.
The URLs changed here are artificial, so we can convert
subject -> topic without breaking anything.
2018-11-14 23:24:06 -08:00
Steve Howell 27d79352da topic -> subject: Extract get_topic_from_message_info().
This changes files where it's safe to just assume caller
may use either "topic" or "subject", and we prefer "topic"
but support "subject".
2018-11-14 23:24:06 -08:00
Steve Howell 44cbcdf00c minor: Remove txt file from lint exception.
It's already ignored because it's not a .py file.
2018-11-14 23:24:06 -08:00
Steve Howell 2505a2f3cc lint: Build exceptions for subject. 2018-11-14 23:24:06 -08:00
Steve Howell a3460d1ef6 lint: Remove obsolete exemptions.
These two files have been cleaned up since the original
lint rule exemption.
2018-11-14 23:24:06 -08:00
Steve Howell c69c40a55a lint: Extract check_file_for_pattern(). 2018-11-14 23:24:06 -08:00
Steve Howell fd0b249a29 lint: Calculate strip_rule outside the loop.
This adds up over a million lines. :)
2018-11-14 23:24:06 -08:00
Steve Howell f386c64759 lint: Speed up by doing fewer regex compiles.
Even though you'd think these regexes would be
cached, compiling the regex outside of looping
through lines makes a difference.

My timings are 8.4s -> 6.0s.  (You need to hack
on the linter to isolate the custom checks.)
2018-11-14 23:24:06 -08:00
Steve Howell 08e44aac0d lint: Extract get_rules_applying_to_fn(). 2018-11-14 23:24:06 -08:00
Steve Howell eda138343b lint: Extract get_line_info_from_file(). 2018-11-14 23:24:06 -08:00
Steve Howell 6d88210110 lint: Extract check_file_for_long_lines().
In passing this removes some unnecessary string
decoding that's a relic of python2.
2018-11-14 23:24:06 -08:00
Steve Howell a942732d83 lint: Get rid of unused skip_rules.
We also go to the modern annotation style for mypy.
2018-11-12 16:00:52 -08:00
Steve Howell 3eb2b2be5f lint: Change lastLine logic slightly. 2018-11-12 15:47:11 -08:00
Steve Howell a86e7fbd9f subject -> topic: Fix message tests. 2018-11-12 15:47:11 -08:00
Steve Howell c90085b28e lint: Extend "subject" prohibition to zerver/tests.
There are several exemptions here, so I triaged
them into various buckets.
2018-11-12 15:47:11 -08:00
Steve Howell d86dd165da gitter/slack/hipchat: Remove "subject" from conversions.
We (lexically) remove "subject" from the conversion code.  The
`build_message` helper calls `set_topic_name` under the hood,
so things still have "subject" in the JSON.

There was good code coverage on `build_message`.
2018-11-12 15:47:11 -08:00
Steve Howell ea98a44db3 webhooks: Replace SUBJECT_WITH_* with TOPIC_WITH_*. 2018-11-12 15:47:11 -08:00
Steve Howell 97062c4017 lint: Prevent use of "subject" in zerver/views. 2018-11-12 15:47:11 -08:00
Eeshan Garg 10570d508f tools/lint: Fix JSON lint pattern for spaces after ':'.
Previously, a string ending in "... 😄" was reported as an
error and the linter complained that there should be a space
after the last ':'. This commit changes the pattern so that the
linter only checks for colons that are preceded by an opening
double-quote (").
2018-11-08 22:07:56 -08:00
Steve Howell cd4d447d90 Minor: change message.subject -> message.topic_name().
This also allows us to lock down
push_notifications.py from using "subject"
in the future.
2018-11-08 16:34:30 +00:00
Steve Howell ff9a6c5ced minor: Rename subject -> topic_name in bugdown. 2018-11-08 16:21:14 +00:00
Steve Howell 70c16ddbcc lint: Prevent "subject" more aggressively.
We now prevent adding "subject" to any code in
zerver/lib, unless you specifically exempt it.

The new set called `FILES_WITH_LEGACY_SUJECT`
is also has comments that give a roadmap of what
to fix.
2018-11-08 15:33:36 +00:00
Steve Howell e802be0fde Remove "subject" in notifications.py.
These last few are all pretty safe.
2018-11-08 15:12:10 +00:00
Steve Howell 226a2826b5 Remove "subject" from zerver/lib/message.py.
We also lock this down file in the linter, so
that you can no longer introduce "subject"
to `message.py`.
2018-11-08 14:45:16 +00:00
Steve Howell 6b6001c46c Remove "subject" from test_classes.py.
We now use "topic" lingo.
2018-11-07 10:03:53 -08:00
Steve Howell 79d5e36ca3 Extract topic_match_sa() helper.
We'll also use this in zerver/views/messages.py, but
that's a bigger change.
2018-11-07 10:03:53 -08:00
Steve Howell df743e8948 lint: Start to deprecate "subject" references. 2018-11-07 10:03:53 -08:00
Rishi Gupta 69e9a8f196 user docs: Add a few portico URLs to documentation crawler test. 2018-11-01 11:59:50 -07:00
Tim Abbott d9887759b4 plans: Fix broken link from FAQ to import docs and add tests. 2018-10-31 11:44:21 -07:00
zegervdv 6dd2218c50 provision: Pass proxy settings through sudo via environment.
This makes provision not fail when installing node behind a proxy.

See #10505 for details.
2018-10-29 13:30:45 -07:00
Vishnu Ks 0cf4a75fb5 coverage: Enforce full test coverage for corporate/views.py. 2018-10-24 19:39:23 -07:00
Anders Kaseorg c3bd293eaf Add shellcheck as a linter.
Fixes #10162.

Tweaked by tabbott to cover documentation as well.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:32:26 -07:00
Anders Kaseorg 49ae9c1e44 pre-commit: Fix shellcheck warnings.
In tools/pre-commit line 18:
if [ -z "$VIRTUAL_ENV" ] && `which vagrant > /dev/null` && [ -e .vagrant ]; then
                            ^-- SC2092: Remove backticks to avoid executing output.
                            ^-- SC2006: Use $(..) instead of legacy `..`.
                             ^-- SC2230: which is non-standard. Use builtin 'command -v' instead.

In tools/pre-commit line 23:
    ./tools/lint --no-gitlint --force $changed_files || true
                                      ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:26:21 -07:00
Anders Kaseorg fc24d2e147 install-aws-server: Fix shellcheck warnings.
In tools/setup/install-aws-server line 25:
zulip_root=${ZULIP_ROOT:-$HOME/zulip}
^-- SC2034: zulip_root appears unused. Verify use (or export if used externally).

In tools/setup/install-aws-server line 40:
if [ -n "$zulip_confdir" ]; then
         ^-- SC2154: zulip_confdir is referenced but not assigned.

In tools/setup/install-aws-server line 55:
VIRTUALENV_NEEDED=$(if $(echo "$type" | grep -q app_frontend); then echo -n yes; else echo -n no; fi)
                       ^-- SC2091: Remove surrounding $() to avoid executing output.

In tools/setup/install-aws-server line 60:
SSH_OPTS=(-o HostKeyAlgorithms=ssh-rsa)
             ^-- SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

In tools/setup/install-aws-server line 69:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

In tools/setup/install-aws-server line 86:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:51:41 -07:00
Anders Kaseorg b15063155c postgres-init-dev-db: Fix shellcheck warnings.
In tools/setup/postgres-init-dev-db line 10:
ROOT_POSTGRES="sudo -i -u "$DEFAULT_USER" psql"
                           ^-- SC2027: The surrounding quotes actually unquote this. Remove or escape them.

In tools/setup/postgres-init-dev-db line 46:
    echo 'ERROR: Try `sudo service postgresql start`?'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

In tools/setup/postgres-init-dev-db line 64:
PGPASS_ESCAPED_PREFIX="*:\*:\*:$USERNAME:"
                         ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".
                            ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:48:57 -07:00
Anders Kaseorg 7f35ad916e django-template-graph: Fix shellcheck warnings.
In tools/django-template-graph line 10:
    for t in $(find -name '*.html' -printf '%P\n'); do
             ^-- SC2044: For loops over find output are fragile. Use find -exec or a while read loop.
               ^-- SC2185: Some finds don't have a default path. Specify '.' explicitly.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg c63d852b55 deploy-branch: Fix shellcheck warnings.
In tools/deploy-branch line 17:
[ $? -ne 0 ] && error_out "Unknown branch: $branch"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 23:
    if [ $? -eq 0 ]; then
         ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 35:
[ $? -ne 0 ] && error_out "Rebase onto origin/master failed"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 39:
[ $? -ne 0 ] && error_out "Push of master to origin/master failed"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg 4a8074000c commit-msg: Fix shellcheck warnings.
In tools/commit-msg line 9:
if [ $(grep '^[^#]' .git/COMMIT_EDITMSG --count) -ne 0 ]; then
     ^-- SC2046: Quote this to prevent word splitting.

In tools/commit-msg line 10:
    lint_cmd="cd ~/zulip && cat \"$1\" | python -m gitlint.cli"
              ^-- SC2089: Quotes/backslashes will be treated literally. Use an array.

In tools/commit-msg line 11:
    if [ -z "$VIRTUAL_ENV" ] && `which vagrant > /dev/null` && [ -e .vagrant ]; then
                                ^-- SC2092: Remove backticks to avoid executing output.
                                ^-- SC2006: Use $(..) instead of legacy `..`.
                                 ^-- SC2230: which is non-standard. Use builtin 'command -v' instead.

In tools/commit-msg line 14:
        $lint_cmd
        ^-- SC2090: Quotes/backslashes in this variable will not be respected.

In tools/commit-msg line 17:
    if [ $? -ne 0 ]; then
         ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg 8b38f61176 clean-branches: Fix shellcheck warnings.
In tools/clean-branches line 33:
                echo -n "Deleting local branch $(echo "$ref" | sed 's!^refs/heads/!!')"
                                                 ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/clean-branches line 41:
                echo -n "Deleting local branch $(echo "$ref" | sed 's!^refs/heads/!!')"
                                                 ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/clean-branches line 49:
                remote_name="$(echo "$ref" | sed 's!^refs/remotes/origin/!!')"
                               ^-- SC2001: See if you can use ${variable//search/replace} instead.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg b5bfe77844 build-release-tarball: Fix shellcheck warnings.
In tools/build-release-tarball line 50:
for i in `cat "$TMPDIR/$prefix/tools/release-tarball-exclude.txt"`; do
         ^-- SC2013: To read lines rather than words, pipe/redirect to a 'while read' loop.
         ^-- SC2006: Use $(..) instead of legacy `..`.

In tools/build-release-tarball line 51:
    rm -r --interactive=never "$TMPDIR/$prefix/$i";
                              ^-- SC2115: Use "${var:?}" to ensure this never expands to / .

In tools/build-release-tarball line 97:
    echo; echo -ne "\033[33mRunning update-prod-static failed. "
                    ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/build-release-tarball line 98:
    echo -e "Check $TMPDIR/update-prod-static.log for more information.\033[0m"
                                                                       ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg a5280d7d02 build-docs: Fix shellcheck warnings.
In tools/build-docs line 3:
cd "$(dirname "$0")"/../docs
^-- SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Rishi Gupta d76f728dda api docs: Rename webhook-walkthrough to incoming-webhooks-walkthrough. 2018-10-16 21:07:00 -07:00
Eeshan Garg 0292ffe04b webhooks/taiga: Add test for when a task is moved between stories.
This commit adds a test for the payload that is generated when
a Task is moved from one user story to another on Taiga's Sprint
Taskboard UI.

This commit also gets up this webhook's test coverage up to 100%.
2018-10-16 13:06:20 -07:00
Aditya Bansal 75ae94e459 font-awesome: Drop support for legacy font awesome icons.
We drop support for usage of `icon-vector` as base class when
including icons from font awesome icons package.
Now on, only icons as specified in font awesome v4.7.0 can be used
in the code base.
2018-10-15 20:14:55 +05:30
Aditya Bansal 7eeed1413c icons: Remove base class from comments and test templates. 2018-10-15 19:55:18 +05:30
Steve Howell b7f764aa29 settings: Add confirm_dialog module.
This module makes it really easy to create are-you-sure
dialogs for dangerous operations.

Basically it's one function with five parameters.  You
give three chunks of HTML, a callback function, and
a parent container.

The first use of this will be in settings_user_groups,
coming up in a couple commits.
2018-10-12 10:37:06 -07:00
Eeshan Garg ba929508e2 webhooks/ifttt: Get test coverage up to 100%.
IFTTT allows custom templating for their payloads, so the onus is
on the user to ensure that their custom templates conform to the
expectations outlined in our IFTTT webhook docs. For that reason,
these payloads weren't generated, but were manually edited.
2018-10-04 12:16:06 -07:00
Eeshan Garg 316f9aa78b webhooks/hellosign: Rewrite the integration from scratch.
After discovering a couple of bugs, I decided to thoroughly test
and rewrite this integration from scratch. The older code wasn't
generating coherent messages.

This also commit gets this integration up to 100% test coverage.
2018-10-04 12:16:06 -07:00
Eeshan Garg 5039f6dfb5 webhooks/gitlab: Get test coverage up to 100%.
Test coverage was improved by removing an unused function and
removing some code (written by me) that was actually handling
Test Hook event types incorrectly.
2018-10-04 12:16:06 -07:00
Eeshan Garg 6e2e2b9125 webhooks/github: Test commit status payloads with target_url.
It was a painful amount of work to generate the actual payload.
Since the only difference was a small build URL, I manually
edited the payload and used that for testing.

This commit gets our GitHub webhook up to 100% test coverage.
2018-10-04 12:16:06 -07:00
Eeshan Garg b40aec3a09 webhooks/freshdesk: Improve test coverage.
Note that Freshdesk allows custom templating for outgoing payloads
in their webhook UI. Therefore, the payloads added in this commit
did not have to be official payloads from Freshdesk.
2018-10-04 12:16:06 -07:00
Eeshan Garg d8101ca139 webhooks/bitbucket2: Improve test coverage.
The lack of coverage was due to:

* An unused function that was never used anywhere.
* get_commit_status_changed_body was using a regex where it didn't
  really need to use one. And there was an if statement that
  assumed that the payload might NOT contain the URL to the commit.
  However, I checked the payload and there shouldn't be any instances
  where a commit event is generated but there is no URL to the commit.
* get_push_tag_body had an `else` condition that really can't happen
  in any payload. I verified this by checking the BitBucket webhook
  docs.
2018-10-04 12:16:06 -07:00
Eeshan Garg 185a023745 webhooks/beanstalk: Improve test coverage.
We shouldn't just ignore exceptions when encoding the incoming
auth credentials. Even if the incoming credentials are properly
encoded, it is better to know when that is the case or if
something else fails.
2018-10-04 12:16:06 -07:00
Harshit Bansal fc7e654d7b emoji: Remove single-digit names for number emojis.
Names like `:1:`, `:2:` etc don't seem to be of much benefit rather
they seem to get rendered at places not intended.
2018-10-02 11:30:11 -07:00
Vishnu Ks 59865797ac billing: Move billing tests from zilencer to corporate. 2018-10-01 15:44:02 -07:00
Vishnu Ks 93b398bc0a billing: Move zilencer/lib/stripe to corporate. 2018-10-01 15:43:44 -07:00
Vishnu Ks daf3b46262 billing: Move billing related templates from zilencer to corporate. 2018-10-01 15:43:39 -07:00
Vishnu Ks 6914ee126c billing: Move billing-related views and urls to corporate. 2018-10-01 15:43:21 -07:00
Tim Abbott 00821d67bb travis: Remove no-longer-present packages from apt-mark hold list.
Apparently, Travis removed the Heroku bundle of packages from their
servers, which made the build start failing when trying to configure
apt to hold their versions (sigh).  This commit removes the
problematic packages.
2018-09-28 14:20:04 -07:00
Greg Price 3b646c5d28 push-to-pull-request: Update local tracking ref, if any.
The companion tool `tools/reset-to-pull-request` has a handy feature
to maintain a local ref tracking the PR: e.g., pr/1234 for PR 1234.
If this were a normal remote-tracking branch maintained by `git fetch`,
it'd get updated on `git push`.  Do the same thing here.

This helps keep a view like `gitk --all @` a bit tidier, by causing
merged PRs to stop pointing at side branches of the main history.
2018-09-25 13:15:02 -07:00
Vishnu Ks 5a6b2ebb1f billing: Add ability for users to change their card. 2018-09-24 19:25:26 -07:00
Anders Kaseorg 5e88b3013a zanitizer_config.pm.sample: keep_file → scrub_filename
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-09-23 20:42:27 -07:00
Eeshan Garg ecd4f821be user docs: Automatically activate correct tab for OS-specific instructions. 2018-09-18 13:49:34 -07:00
Vishnu Ks d6b548bd57 styles: Extract billing SCSS into its own stylesheet. 2018-09-11 00:08:38 -07:00
Steve Howell 06225d1424 tests: Clean up calls to tools/webpack.
Before this change, the way we loaded
webpack for various tools was brittle.

First, I addressed test-api and test-help-documentation.

These tools used to be unable to run standalone on a
clean provision, because they were (indirectly)
calling tools/webpack without the `--test` option.

The problem was a bit obscure, since running things
like `./tools/test-backend` or `./tools/test-all` in
your workflow would create `./var/webpack-stats-test.json`
for the broken tools (and then they would work).

The tools themselves weren't broken; they were the
only relying on the common `test_server_running` helper.
And even that helper wasn't broken; it was just that
`run-dev.py` wasn't respecting the `--test` option.

So I made it so that `./tools/run-dev` passes in `--test` to
`./tools/webpack`.

To confuse matters even more, for some reason Casper
uses `./webpack-stats-production.json` via various
hacks for its webpack configuration, so when I fixed
the other tests, it broke Casper.

Here is the Casper-related hack in zproject/test_settings.py,
which was in place before my change and remains
after it:

    if CASPER_TESTS:
        WEBPACK_FILE = 'webpack-stats-production.json'
    else:
        WEBPACK_FILE = os.path.join('var', 'webpack-stats-test.json')

I added similar logic in tools/webpack:

    if "CASPER_TESTS" in os.environ:
        build_for_prod_or_casper(args.quiet)

I also made the helper functions in `./tools/webpack` have
nicer names.

So, now tools should all be able to run standalone and not
rely on previous tools creating webpack stats files for
them and leaving them in the file system.  That's good.

Things are still a bit janky, though.  It's not completely
clear to me why `test-js-with-casper` should work off of
a different webpack configuration than the other tests.

For now most of the jankiness is around Casper, and we have
hacks in two different places, `zproject/test_settings.py` and
`tools/webpack` to force it to use the production stats
file instead of the "test" one, even though Casper uses
test-like settings for other things like which database
you're using.
2018-09-07 11:39:55 -04:00
Steve Howell 25ad755914 Fix usage for test-api. 2018-09-07 11:39:55 -04:00
Steve Howell 336c2cee20 Revert unused code for creating non-editable pills.
We don't use input.create_non_editable_pill() in our
code yet.  If we add this back, we'll want to have node
tests on it.

Removing this unused code brings us to 100% line
coverage for input_pill.js.

This directly reverts 5c11ab85 with the small addition
of adding input_pill to our list of fully covered
modules.
2018-08-31 16:26:05 -07:00
Harshit Bansal 88bfb9778b build_emoji: Add infrastructure for google-blob and twitter emojisets. 2018-08-31 11:52:24 -07:00
Marco Burstein 25e624eab1 team: Include contributions from legacy repositories in the total.
These repositories (`zulip-ios-legacy` and `zulip-android`) are
deprecated, and as such should not have their own tabs, but still
should be included in the total contributions count.
2018-08-28 16:45:00 -07:00
Tim Abbott 2c7f9ce0fc puppet: Fix puppet-lint warnings in various manifests.
Apparently, `puppet-lint` on Ubuntu trusty throws warnings for certain
quoting patterns that are OK in modern `puppet-lint`.  I believe the
old Zulip code was actually correct (i.e. the old `puppet-lint`
implementation was the problem), but it seems worth changing anyway to
suppress the warnings.

We also exclude more of puppet-apt from linting, since it's
third-party code.
2018-08-28 13:46:31 -07:00
Lyla Fischer 1efcdfdb5c templates: Merge Help and API doc main into documentation_main.
With minor fixes by eeshangarg.
2018-08-27 20:19:24 -02:30
Harshit Bansal e204fe3948 minor: Fix for failing tests. 2018-08-27 06:16:16 +00:00
Harshit Bansal d96d6a96f7 build_emoji: Add a missing check for image existence. 2018-08-26 22:16:23 -07:00
Harshit Bansal f5454dff51 build_emoji: Automatically calculate sprite dimensions.
Instead of using a hardcoded value for spritesheet dimensions,
automatically calculate it using `emoji_data`. This will free
us from updating it only emoji datasource update as well as
allow us to add google blob emojiset.
2018-08-26 22:16:23 -07:00
Harshit Bansal dbc3774c2f build_emoji: Refactor `get_emoji_code()` to not fail for old datasources.
If `non_qualified` field is not present then return the value of
`unified` field instead of failing.
2018-08-26 22:16:23 -07:00
Harshit Bansal ec2ebd1cc9 build_emoji: Change sprite CSS files to use hyphens in names.
This is more consistent with our other names in emoji infra.
2018-08-26 22:16:23 -07:00
Harshit Bansal 9bd56d0ce4 build_emoji: Change emoji sprite sheets to use hyphens in names.
Use of hyphens is more consistent with our other emoji infra names.
2018-08-26 22:16:23 -07:00
Harshit Bansal aa6f098496 build_emoji: Extract `setup_emoji_farm()`. 2018-08-26 22:16:23 -07:00
Harshit Bansal dc9ff928f3 build_emoji: Refactor `ensure_emoji_image()` to take paths as args. 2018-08-26 22:16:23 -07:00
Harshit Bansal ed694001da build_emoji: Rename `setup_emoji_farm()` to `setup_emoji_farms()`. 2018-08-26 22:16:23 -07:00
Steve Howell ca272ef49d tests: Test how we handle extra data for zform messages. 2018-08-24 10:00:25 -07:00
Steve Howell bee592b44e node tests: Bring list_cursor to 100% coverage.
We had some indirect coverage of this already, so
this mostly focuses on error cases and corner
cases.
2018-08-24 10:00:04 -07:00
Steve Howell 44756a326d tests: Test duplicate default stream/groups.
This also gets us to 100% line coverage
for actions.py.
2018-08-22 16:52:30 -07:00
Steve Howell 4318f75718 Remove `code` from JsonableError.__init__.
Since this class was built, folks have always chosen
to subclass JsonableError for situations where
the default of ErrorCode.BAD_REQUEST is insufficient.

So now we simplify the use cases, which also gets
us 100% coverage on this core module.
2018-08-22 16:51:40 -07:00
Tim Abbott b93d5bd5e8 test-js-with-node: Temporarily remove server_events_dispatch.js.
This fixes master failing tests since 2aeabf24a6.
2018-08-21 14:40:05 -07:00
Joshua Pan 793ec9e266 tests: Test all update_display_settings events. 2018-08-19 14:51:31 -04:00
xificurC 9e053c74cf scripts: Allow configuring a custom CA bundle for build process.
For building Zulip in an environment where a custom CA certificate is
required to access the public Internet, one needs to be able to
specify that CA certificate for all network access done by the Zulip
installer/build process.  This change allows configuring that via the
environment.
2018-08-13 15:45:28 -07:00
Tim Abbott 90828297e4 puppet-lint: Enforce double_quoted_strings check.
This makes our puppet codebase more consistent by using single-quoted
strings consistently.
2018-08-13 12:31:19 -07:00
Tim Abbott d0b51b70f4 puppet-lint: Enforce 2sp_soft_tables puppet-lint check.
This cleans up the puppet codebase's whitespace formatting to be more
consistent.
2018-08-13 12:31:16 -07:00
Tim Abbott b26e0a957d puppet-lint: Enforce arrow_alignment check.
This fixes all exceptions in our puppet codebase to this lint rule.
2018-08-13 12:30:57 -07:00
Aditya Bansal a3b8ab46b3 puppet-lint: Include in linter tool chain to run by default.
We start to use puppet-lint to lint puppet modules by default by
adding it to tools/lint (which controls our linter tool chain).

We also define a few puppet-lint rules to exclude.

Fixes: #9185.
2018-08-13 12:30:57 -07:00
Aditya Bansal 56d49980de puppet-lint: Add as a dependancy to install during provision process. 2018-08-13 12:30:57 -07:00
Harshit Bansal 29daf76420 emoji: Don't generate emoji farms corresponding to now banned emojisets.
This will help us in reducing the size of the release tarball
significantly. I have refrained from changing the `EMOJISETS`
constant in the `emoji_setup_utils.py` as that controls the
emojisets that we want to support. Since we want to re-enable
the feature of changing emojisets sometime again in the future
that variable should be kept as it is as it controls several
other things like emoji scripts that we use to generate emoji
names. Changing it might cause hard to catch bugs.
2018-08-13 12:09:06 -07:00
Harshit Bansal 36b977c382 emoji: Refactor `generate_sprite_css_files()` to take emojiset in args. 2018-08-13 12:09:06 -07:00
Harshit Bansal 67bbbbb007 emoji: Move logic for hard-coding octopus emoji.
This logic was incorrectly placed in `setup_old_emoji_farm()` while
it should have been placed in `setup_emoji_farm()`.
2018-08-13 12:09:06 -07:00
Harshit Bansal 1eb3825048 emoji: Remove now-unnecessary `get_remapped_emojis_map()`. 2018-08-13 12:08:14 -07:00
Harshit Bansal 99e1a81055 emoji: Fix some emoji images not loading in missed message emails.
`emoji-datasource` package v4.0.4 introduced the concept of qualified
and non-qualified emoji codes. As chat programs don't need to use
emoji representation selector, so we used migrated our infrastructure
to use non-qualified emoji codes. But we missed the fact that the
emoji file names in emoji farm are based on emoji data's 'unified'
field and the value of this field has changed. Consequently the image
file names must also have been changed. We used `emoji_code` while
converting the span tags to img tags while processing notifications.
But since now `emoji_code` refers to non-qualified code while image
file names are based on qualified code, we need to rename images
to correctly do the conversion. This commit just fixes this.
2018-08-13 12:03:16 -07:00
Steve Howell 413a0174f4 Extract a zephyr.py library.
Right now it only has one function, but the function
we removed never really belonged in actions.py, and
now we have better test coverage on actions.py, which
is an important module to get to 100%.
2018-08-11 14:51:26 -07:00
Max Nussenbaum 570b9515c6 billing: Add location for errors on billing page.
This adds a location for error messages to the billing page,
using the standard error styling.
2018-08-10 22:48:10 -07:00
Vishnu Ks 0c84260faa billing: Enforce full coverage in zilencer/tests. 2018-08-10 13:52:35 -07:00
Vishnu Ks 9946202148 coverage: Enforce full coverage for zilencer/lib/stripe.py. 2018-08-10 13:52:35 -07:00
Tim Abbott aaedec1fdb tools: Move a few i18n scripts to tools/i18n.
These are likely to only be run by Zulip maintainers.
2018-08-09 15:46:27 -07:00
Tim Abbott ba626dcad6 tools: Move optimize-svg to tools/setup/. 2018-08-09 15:39:57 -07:00
Tim Abbott c2e5cc99a4 tools: Move generate-custom-icon-webfont to a subdirectory. 2018-08-09 15:39:57 -07:00
Tim Abbott 2bb04fc720 tools: Remove send-github-payloads.
This ancient tool predates our practice of collecting test fixtures
for third-party integrations, which is a better general system for the
problem this solved.
2018-08-09 15:39:57 -07:00
Tim Abbott 74dc3e9a3e tools: Remove unused munge-postgres-logs.
This hasn't been used in several years.
2018-08-09 15:39:57 -07:00
Tim Abbott bd945333a3 tools: Remove replace-tarball-shebang.
This was a temporary helper tool for part of
tools/build-release-tarball during our Python 3 migration, which
finished some time ago.
2018-08-09 15:27:09 -07:00
Tim Abbott 78539e7819 tools: Remove unused deployment-lock-ctl.
This interacted with a deployment locking system that is no longer
part of the project.
2018-08-09 15:25:15 -07:00
Rhea Parekh 26300110ca import: Fix rendered_content in imported messages.
After the messages have been imported, set the rendered_content of the
messages instead of leaving its value to be 'None'.

This is important to ensure that:
(1) Performance for users is good after completing the import.
(2) The database's full-text indexes have all of the imported messages
(which only happens properly when Message rows have their
rendered_content field edited).

Fixes #9168.
2018-08-09 15:12:53 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Yago González a0d8cbc9cb test-backend: Fix typo. 2018-08-08 16:34:29 -07:00
Cynthia Lin ecbc58c586 settings: Fix color styling for nested inputs to work in night mode. 2018-08-08 11:29:48 -07:00
Rhea Parekh 30cc7354eb import: Move 'process_avatars' to import_util. 2018-08-07 16:45:40 -07:00
Rhea Parekh 1117455a90 import: Move 'ZerverFieldsT' and 'build_zerver_realm' to import_util. 2018-08-07 16:35:56 -07:00
Vishnu Ks 119d108da0 droplets: Update code and remove nvm settings from bashrc.
* Synced zulip repo to master and provisioned
* Synced python-zulip-api repo to master and provisioned.
* Removed unnecessary nvm config from bashrc
2018-08-07 13:38:18 -07:00
Tim Abbott 4300683bca lint: Fix mypy error in check_pyflakes annotations.
Introduced in 78a93b8d9e.
2018-08-07 09:59:14 -07:00
Tim Abbott e67fa36803 lint: Update config to split files more evenly. 2018-08-07 09:42:37 -07:00
Tim Abbott 78a93b8d9e lint: Refactor pyflakes to use more consistent style.
In particular, we no longer pass in the full `by_lang` object and
expect it to pull out the Python piece.
2018-08-07 09:42:37 -07:00
Tim Abbott 458c17868f zulint: Move pycodestyle runner code to library.
This way, only the Zulip project's specific configuration ends up
living in pep8.py.
2018-08-04 19:54:18 -07:00
Tim Abbott 3b53ddf779 zulint: Stop importing color codes from zulip_tools.py.
This is I think the final dependency that zulint/ has on the rest of
the project.
2018-08-04 19:53:53 -07:00
Tim Abbott c0beeebc3e zulint: Move printer.py module to tools/zulint.
At this point, tools/linter_lib is intended to be the directory of
Zulip-specific code.
2018-08-04 19:53:53 -07:00
Tim Abbott 55ba434ecb zulint: Extract LinterConfig class to zulint. 2018-08-04 19:53:53 -07:00
Tim Abbott f74bb8a9b2 zulint: Extract run_parallel and do_lint. 2018-08-04 19:53:53 -07:00
Tim Abbott acf8ec492d zulint: Extract common linter argument parsing logic.
The remaining code is Zulip-specific business logic.
2018-08-04 19:53:53 -07:00
Tim Abbott 1fcc2a6ea4 zulint: Move lister.py to tools/zulint.
This is preparatory refactoring work for being able to extract the
linter as an external project called "zulint".
2018-08-04 19:53:53 -07:00
Joshua Pan 5303281dfb tests: Fully cover hash_util.js. 2018-08-04 17:56:02 -07:00
Abhilash Verma fd9652a9d7 settings_emoji: Make strings more user-friendly. 2018-08-04 09:37:48 -07:00
Steve Howell c8898e1dc8 refactor: Move by_conversation_and_time_uri to hash_util.
This removes the 100% coverage on hash_util, but we are
pretty careful about not messing with this code.
2018-08-04 09:32:27 -07:00
Anders Kaseorg d79555e9be update-locked-requirements: Fix shellcheck warnings.
In tools/update-locked-requirements line 66:
compile_requirements requirements/prod.in $OUTPUT_BASE_DIR/prod.txt
                                          ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 67:
compile_requirements requirements/dev.in $OUTPUT_BASE_DIR/dev.txt
                                         ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 68:
compile_requirements requirements/mypy.in $OUTPUT_BASE_DIR/mypy.txt
                                          ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 69:
compile_requirements requirements/docs.in $OUTPUT_BASE_DIR/docs.txt
                                          ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 70:
compile_requirements requirements/thumbor.in $OUTPUT_BASE_DIR/thumbor.txt py2
                                             ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg 3e7bf9be9c production-helper: Fix shellcheck warnings.
In tools/travis/production-helper line 24:
if ! apt-get dist-upgrade -y $APT_OPTIONS; then
                             ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/travis/production-helper line 26:
    apt-get dist-upgrade -y $APT_OPTIONS
                            ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg e05a56aa8b test-migrations: Fix shellcheck warnings.
In tools/test-migrations line 18:
    echo "$new_auto_named_migrations" | sed 's/\[[x ]\] /  /'
    ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/test-migrations line 27:
    echo 'ERROR: Migrations are not consistent with models!  Fix with `./tools/renumber-migrations`.'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg c4ed4bc021 test-install: Fix shellcheck warnings.
In tools/test-install/destroy-all line 31:
  | while read c
          ^-- SC2162: read without -r will mangle backslashes.

In tools/test-install/install line 57:
    installer_dir="$(readlink -f $INSTALLER)"
                                 ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/test-install/lxc-wait line 30:
for i in {1..60}; do
^-- SC2034: i appears unused. Verify use (or export if used externally).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg e1d770d451 test-documentation: Fix shellcheck warnings.
In tools/test-documentation line 6:
    echo -e "\e[${color_code}m${message}\e[0m" >&2
             ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".
                                        ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".

In tools/test-documentation line 41:
scrapy crawl_with_status documentation_crawler $loglevel
                                               ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg bb51503907 test-all-docker: Fix shellcheck warnings.
In tools/test-all-docker line 7:
source /home/zulip/.bash_profile
^-- SC1091: Not following: /home/zulip/.bash_profile: openBinaryFile: does not exist (No such file or directory)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg c93f53b463 test-all: Fix shellcheck warnings.
In tools/test-all line 7:
TEMP=`getopt -o f --long force -- "$@"`
     ^-- SC2006: Use $(..) instead of legacy `..`.

In tools/test-all line 24:
    echo "Running $@"
                  ^-- SC2145: Argument mixes string and array. Use * or separate argument.

In tools/test-all line 26:
        printf "\n\e[31;1mFAILED\e[0m $@\n"
               ^-- SC2059: Don't use variables in the printf format string. Use printf "..%s.." "$foo".
                ^-- SC1117: Backslash is literal in "\n". Prefer explicit escaping: "\\n".
                  ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".
                                ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".
                                      ^-- SC2145: Argument mixes string and array. Use * or separate argument.
                                        ^-- SC1117: Backslash is literal in "\n". Prefer explicit escaping: "\\n".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg e9656564c2 start-dockers: Fix shellcheck warnings.
In tools/start-dockers line 7:
source /home/zulip/.bash_profile
^-- SC1091: Not following: /home/zulip/.bash_profile: openBinaryFile: does not exist (No such file or directory)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg f4c4e857ff generate-test-credentials: Fix shellcheck warnings.
In tools/setup/generate-test-credentials line 10:
password=$(./manage.py print_initial_password "$email" | fgrep "$email" | awk '{ print $2 }')
                                                         ^-- SC2197: fgrep is non-standard and deprecated. Use grep -F instead.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg af64bff9d4 reset-to-pull-request: Fix shellcheck warnings.
In tools/reset-to-pull-request line 25:
    git fetch "$remote" +"pull/$request_id/head":"$target_ref"
                                                ^-- SC2140: Word is of the form "A"B"C" (B indicated). Did you mean "ABC" or "A\"B\"C"?

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg c669626666 provision: Fix shellcheck warnings.
In tools/provision line 13:
FAIL="\033[91m"
      ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/provision line 14:
WARNING="\033[93m"
         ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/provision line 15:
ENDC="\033[0m"
      ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/provision line 19:
PARENT_PATH=$( cd "$(dirname "${BASH_SOURCE}")" ; pwd -P )
                              ^-- SC2128: Expanding an array without an index only gives the first element.

In tools/provision line 32:
if [ $failed = 1 ]; then
     ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/provision line 49:
    echo 'or just close this shell and start a new one (with Vagrant, `vagrant ssh`).'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg d7c7e8dd7c optimize-svg: Fix shellcheck warnings.
In tools/optimize-svg line 3:
if [ `node_modules/.bin/svgo -f static/images/integrations/logos | grep -o '\.[0-9]% = ' | wc -l` -ge 1 ]
     ^-- SC2046: Quote this to prevent word splitting.
     ^-- SC2006: Use $(..) instead of legacy `..`.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg cd451491ff find-unused-css: Fix shellcheck warnings.
In tools/find-unused-css line 5:
    if [ $(git grep "$n" | grep -v '^static/styles/zulip.css' | wc -l) -eq 0 ]; then
         ^-- SC2046: Quote this to prevent word splitting.
                           ^-- SC2126: Consider using grep -c instead of grep|wc -l.

In tools/find-unused-css line 6:
        echo $n
             ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg b920d19358 fetch-rebase-pull-request: Fix shellcheck warnings.
In tools/fetch-rebase-pull-request line 15:
git checkout -B "review-${request_id}" $remote/master
                                       ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg e7e25fb70b do-destroy-rebuild-test-database: Fix shellcheck warnings.
In tools/do-destroy-rebuild-test-database line 6:
"`dirname "$0"`/../tools/setup/generate-fixtures" --force
 ^-- SC2006: Use $(..) instead of legacy `..`.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Rhea Parekh ee37866687 import: Add gitter import file in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Rhea Parekh b8e1e8b31d import: Add slack import files in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Anders Kaseorg 510c97d861 scripts: Use shell quoting when displaying commands to be run.
This way, commands with arguments containing whitespace or
metacharacters are unambiguously readable.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:08 -07:00
Anders Kaseorg de5e3d3fbd tools/review: Don’t pretend to emulate shell=True either.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:08 -07:00
Anders Kaseorg 9a99d76296 tools: Remove pointless uses of shell=True.
Which, not coincidentally, is all of them.  Don’t use shell=True.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:06 -07:00
Steve Howell 95d136ca5e Extract search_util.js module.
We probably should have done this a while ago, even
though these functions are pretty tiny.  The goal here
is to make it easier to have more consistent search
semantics.

Our first use case is subs.js.  In this case we
are able to decouple a bit of generic string
matching from the subs-specific code.
2018-07-30 11:25:32 -07:00
Greg Price 4ba8015002 tools/reset...: Add experimental option to record the PR as a ref.
I often find myself looking manually through the reflog of `master` to
find a commit I previously reset to with tools/reset-to-pull-request .
Sometimes I want to see a previous version of a PR I'm reviewing a
revised version of; sometimes to look at two related PRs together.

So, here's a feature to automate that by saving each PR branch in its
own ref, with a name like `refs/remotes/pr/1234` -- or `pr/1234`, as
you'd normally refer to it.

To enable this, set the new config option:
  $ git config zulip.prPseudoRemote pr
(Or you can pick another name.)

The reason I hesitate to just make this the behavior for everyone
immediately is that the resulting `pr/1234` refs will naturally
accumulate and may clutter up the view -- and because with the
`refs/remotes/` style of name I've chosen, it requires a bit of
Git plumbing to clean them up.  (Use `git update-ref -d`.)
I'll play with it and iterate; comments welcome from other willing
early adopters.
2018-07-27 23:47:32 -07:00
Harshit Bansal cf5b2b4815 emoji: Change emoticon mapping for `:)`, `(:` and `:(`.
See discussion on CZO:
https://chat.zulip.org/#narrow/stream/101-design/subject/emoji.20picker/near/617811
2018-07-26 11:17:03 -07:00
Anders Kaseorg 19f37fb3d4 tools/test-queue-worker-reload: Avoid shelling out for touch.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Anders Kaseorg d3f8f74ac4 tools/test-locked-requirements: Avoid shelling out for cp, mkdir.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Anders Kaseorg dfed7da911 tools/lib/provision.py: Avoid shelling out for mkdir, touch.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Aditya Bansal 0042694e24 stylelint: Start using stylelint to lint our css.
Tweaked by tabbott to document it.

Fixes #8894.
2018-07-25 15:03:35 -07:00
Priyank Patel 8a012b9e7f Revert "webpack: Combine archive-style bundle into archive."
This reverts commit d1d9d63841, and fixes
the fonts in webapp being overwritten by bootstarap due to webpack bundling.
2018-07-24 16:00:26 -07:00
Greg Price 79dfe6e0ae push-to-pull-request: Fix shell usage in an error path.
Errors go to stderr, not stdout.

While we're here, match the message itself to the others in
this script.
2018-07-23 23:19:27 -07:00
Greg Price 26d0904436 push-to-pull-request: Give a nice error if `jq` missing. 2018-07-23 23:19:27 -07:00
Priyank Patel d1d9d63841 webpack: Combine archive-style bundle into archive. 2018-07-23 15:19:47 -07:00
Priyank Patel f8398238a9 webpack: Migrate archive bundle to webpack.
Tested by visiting localhost:9991/archive/streams/1/topics/denmark3.
2018-07-23 15:19:44 -07:00
Harshit Bansal c0b0fb7cce emoji: Move `EMOTICON_CONVERSIONS` mapping to build_emoji infra.
This commit closes a long pending issue which involved moving the
`EMOTICON_CONVERSION` mapping to build_emoji infrastructure so
that there is only one source of truth. This was pending from the
time when this feature was implemented.
2018-07-23 12:35:08 -07:00
Joshua Pan 565fd75661 tests: Test closing alert word status msg. 2018-07-23 14:53:49 -04:00
Tim Abbott 2918cc92f3 search: Duplicate HTML template content for search area.
This is preparation for being able to work on the search pills feature
without making any user-facing changes until we're ready to enable it.
2018-07-23 11:29:10 -07:00
Rhea Parekh 3a6921e54b gitter import: Add gitter data conversion script. 2018-07-23 08:36:30 -07:00
Harshit Bansal bf70955c0c emoji: Update `emoji-datasource` packages.
This commit updates the `emoji-datasource` packages to version 4.0.4.
This update brings following changes to emoji infra:

1: Fix for the bleeding sprite sheets.

2: The category of some emojis has been changed. Categorywise breakup of
   net gain or loss is as follows:
    Travel & Places: 58 (gain)
    Symbols: 47 (loss)
    Smileys & People: 52 (gain)
    Objects: 11 (loss)
    Food & Drink: 3 (gain)
    Animals and Nature: 46 (gain)
    Activities: 9 (loss)

3: There were some changes in the image farm of the package which were
   breaking our old emoji farm. I fixed them by modifying the remapped
   emoji map.

Fixes: #8235.
2018-07-23 08:04:58 -07:00
Harshit Bansal 6c28a60db0 emoji: Hardcode octopus emoji.
Google emojiset's octopus is really cute and whole Zulip community
loves it. So using a CSS hack, we hardcode octopus emoji to use image
from Google's emojiset only irrespective of the choosen emojiset.
2018-07-23 08:04:58 -07:00
Harshit Bansal 3f73826ffa emoji: Extract `get_emoji_code()`. 2018-07-23 08:04:58 -07:00
Harshit Bansal 69eaa2de67 emoji: Move `remapped_emojis` list to emoji_setup_utils.py. 2018-07-23 08:04:58 -07:00
Joshua Pan 015bc8b01e tests: Reach 100% coverage for keydown_util.
This commit pretty much just gets line coverage.
2018-07-23 08:42:11 -04:00
Joshua Pan 0596632457 lint: Exclude frontend_tests/node_tests from $().text rule. 2018-07-20 12:13:56 -07:00
Steve Howell b9c9dd9b5a lint: Turn off the CSS checker (officially).
Our CSS checker globs for .css files.  Since the
SCSS cutover, it has been a no-op, so there's no
sense launching it.  See #8894 for details on
future plans.
2018-07-20 10:03:28 -04:00
Anders Kaseorg 8698198452 tools/update-authors-json: Avoid shelling out for cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 6a8f0165ee tools/test-help-documentation: Avoid shelling out for mkdir.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg f01942e510 tools/test-emoji-name-scripts: Avoid shelling out for cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 07f5b81406 tools/minify-js: Avoid shelling out for mkdir, cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 2d12b5b3d9 tools/diagnose: Avoid shelling out for pwd.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Tim Abbott 811ac718d6 build_emoji: Document a quirk of generate_sprite_css_files. 2018-07-13 21:19:26 +05:30
Harshit Bansal ec1ce08523 emoji_setup_utils: Migrate to python3 type annotations. 2018-07-13 21:19:26 +05:30
Harshit Bansal f6fe5d6266 build_emoji: Migrate to python3 type annotations. 2018-07-13 21:19:25 +05:30
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Tim Abbott 7a86e83a6f build_emoji: Use with for writing to files.
This is a nice code cleanup.
2018-07-13 21:07:08 +05:30
Tim Abbott 85243c525d build_emoji: Pass around emoji_map data structure less.
It wasn't used in either of these functions.
2018-07-13 21:04:37 +05:30
Harshit Bansal b9f974ba61 tools: Add a tool to validate emoji name listing scripts.
This tools emoji name import/export tooling by verifying that no
information gets modified in a round-trip conversion.
2018-07-13 20:53:25 +05:30
Harshit Bansal 628e868d1e tools: Add `export_emoji_names_to_csv`.
This tool is used for exporting `emoji_names.py` to a csv file.
2018-07-13 20:53:08 +05:30
Harshit Bansal 05f85eb94d emoji: Add emoji_names.py file.
Credits to @rishig, Alice Lai, and @rntharu for naming all the emoji.

Names are inspired by iamcal, gemoji, and unicode names, sources like
emojipedia and iemoji, google search results for articles about emoji,
and emoji usage on twitter.
2018-07-13 20:53:07 +05:30
Harshit Bansal ae17a15b29 tools: Add generate_emoji_names_table tool.
This tool generates a listing of the emojis as per the current cache
in use.
2018-07-13 20:53:07 +05:30
Harshit Bansal fef5b43e22 minor: Fix directory name label in 'import_emoji_names_from_csv'. 2018-07-13 20:53:02 +05:30
Max Nussenbaum 8fd900626d billing: Style billing page.
This adds styles to /billing.
2018-07-13 18:30:38 +05:30
Tim Abbott 3cc93fd721 setup_venv: Fix missing libssl-dev dependency.
We were already correctly including libssl-dev in Zulip's dependencies
in development environment provisioning, but (at least now) it's
needed to build certain Python packages like pycurl when building a
Zulip virtualenv in production.  I haven't investigated why we didn't
need this on Ubuntu, but one possible reason would be that some other
library in our dependencies list happens to depend on it on Ubuntu.

We fix this by moving the dependency over to the shared
VENV_DEPENDENCIES list.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Harshit Bansal 8ada7cfe5b tools: Add import_emoji_names_from_csv tool. 2018-07-12 18:05:02 +05:30
Greg Price 36cf898589 mypy: Add a place to put type stubs.
This will allow us to begin to add our own stubs for external
libraries.  Writing stubs can be surprisingly little work to do, and
can have high leverage in keeping our type annotations high-quality.
2018-07-12 14:10:09 +05:30
Yago González a3d42d9901 test-api: Pass non-admin client to the test helpers. 2018-07-11 23:54:00 +05:30
Cynthia Lin 7d9281fa6c simplebar: Add simplebar dependency for help page scrolling. 2018-07-11 20:04:55 +05:30
Puneeth Chaganti 3dace27ed6 tools: Only files starting with same migration number conflict. 2018-07-10 21:09:34 +05:30
Shayan Toqraee 0757d022f5 messages: Add support for right-to-left messages.
This implements right-to-left message automatic detection support in
the compose box as well as the message feed.  Full unit tests and
support in the message-editing UI are for future work (as are
potentially more fancy things like supporting things like
right-to-left multi-word names for users/streams/etc.).

Fixes #3123.
2018-07-10 10:47:56 +05:30
Rishi Gupta 16334a1ba7 billing: Update billing system. 2018-07-09 14:33:07 +05:30
Aditya Bansal b6ba162769 generate-fixtures: Extract zulip_test_template creation as a function.
This will be helpful in the upcoming changes which will make use
of this extracted function to re-create zulip_test_template after
migrating zulip_test db so that we have latest schema in tests.
2018-07-09 09:11:48 +05:30
Armaan Ahluwalia ba6543b850 webpack: Fix missing exposed modules left out of transition.
This commit fixes some modules that were erroneously left out while
transitioning app.js to webpack. This commit exposes them using
expose-loader or setting them directly to window.
2018-07-07 08:00:34 +02:00
Shubham Dhama ced6f967bc voting_widget: Increase node test coverage to 100%. 2018-07-06 11:30:12 -04:00
Tim Abbott 0bd139af0c webpack: Fix missing expose for toMarkdown. 2018-07-05 13:24:19 +02:00
Armaan Ahluwalia 54d3d8e8b3 webpack: Transition app.js to be compiled by webpack.
This commit moves all files previously under the 'app' bundle in
the Django pipeline to being compiled by webpack under the 'app'
entry point. In the process, it moves assets under the app entry
to a file called app.js that consumes all relevant css and js files.

This commit also edits the webpack config to be able to expose certain
variables for third party libraries that are currently required by
some modules. This is bad coding form and should be refactored to
requiring whatever dependencies a module may have; we're just
deferring that to the future to simplify the series of transitions we
need to do here. The variable exposure is done using expose-loader in
webpack.

The app/index.html template is edited to override the newly introduced
'commonjs' block in the base template. This is done as a temporary
measure so as not to disrupt other pages on the app during the transition.

It also fixes the value of the 'this' context that was being inferred
as window by third party libraries. This is done using imports-loader
in the webpack config.  This is also messy and probably isn't how we
want things to work long term.
2018-07-05 11:03:08 +02:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Shubham Dhama b938523592 widgets: Increase node tests coverage to 100%. 2018-07-03 08:48:49 -04:00
Yashashvi Dave 2b478268f3 third: Add sortablejs library. 2018-07-01 02:05:00 -07:00
Vishnu Ks e1bf749d3c provision: Run build_pygments_data only if required. 2018-06-23 00:15:39 +05:30
Vishnu Ks 109fa85614 provision: Rename file_hash_updated to file_or_package_hash_updated.
Check for changes in package version as well along
with the files.
2018-06-22 23:40:31 +05:30
Vishnu Ks 431d9d6076 provision: Run generate-custom-icon-webfont only if required. 2018-06-22 23:40:31 +05:30
Shubham Padia 6853530eba search: Add search_pill.js with basic operations.
Adds search_pill.js to the static asset pipeline. The items
for search pill contain 2 keys, display_value and search_string.
Adding all the operator information i.e the operator, operand and
negated fields along with the search_string and description was tried out.
It was dropped because it didn't provide any advantage as one had to
always calculate the search_string and the description from the operator.
2018-06-22 09:05:36 -04:00
Shubham Dhama c167227ddc search: Add 100% coverage for search.js. 2018-06-22 07:58:06 -04:00
Greg Price 0462e85d02 tools: Make remote configurable in {reset,push}-to-pull-request.
The appropriate name for the remote pointing at the repo we maintain
may be `upstream` for most of our repos... but not when we're
downstream of someone else, e.g. for react-native.  So, make it easy
to configure per-repo.
2018-06-21 17:28:21 -07:00
Vishnu Ks 59169cc302 provision: Run inline-email-css only if required.
This saves a few hundred milliseconds on provision time.

Edited by tabbott to include email.css in the dependencies.

Fixes #9468.
2018-06-18 06:58:55 -07:00
Vishnu Ks 7b8e79ae48 provision: Refactor hashing of compilemessages into a function.
This allows it to be reused for other tools.

Edited by tabbott to remove the use of "compilemessages" in variable
names.
2018-06-18 06:55:36 -07:00
Raymond Akornor e82581e5bd provision: Close opened files automatically. 2018-06-12 13:38:58 -04:00