Commit Graph

28658 Commits

Author SHA1 Message Date
Tim Abbott 39ea471cf1 error reports: Ensure we filter API keys from query strings.
For some webhook endpoints where the third-party API requires us to do
this, the user's API key might appear in error emails through
appearing in the `QUERY_STRING` parameter.  Fix that by filtering any
actual content from those; what we usually need for debugging is just
what set of parameters were provided.
2018-10-19 15:03:14 -07:00
Yashashvi Dave cc810f8951 settings: Fix width not maximum while uploading user avatar.
New user avatar width is not maximum when user upload
new image. Because wrong html element is accessed for
setting value of image src attribute.
This commit removes these code from success of ajax call,
cause we already handle this in event `user_events - avatar_url`.
2018-10-19 12:36:18 -07:00
Yashashvi Dave 41fbb16cdf org settings: Fix error when admin update realm users full names.
Currently, if there is only one admin in realm and admin tries
to updates any non-adminuser's full name it throws error,
"Cannot remove only realm admin". Because in `/json/users/<user_id>`
api check_if_last_admin_is_changed is checked even if property
is_admin is not changed.

This commit fix this issue and add tests for it.
2018-10-19 12:34:20 -07:00
Rishi Gupta 97ed71ca69 portico: Reorder quotes in carousel on /hello. 2018-10-18 16:31:03 -07:00
Rishi Gupta aad5c3df48 user docs: Fix typo causing broken gear icon. 2018-10-18 16:30:42 -07:00
Rishi Gupta f14ed03707 docs: Update import instructions in export-and-import. 2018-10-18 16:22:31 -07:00
Tim Abbott c3c83d7e83 docs: Document PROVISION_VERSION. 2018-10-18 15:38:55 -07:00
Anders Kaseorg c3bd293eaf Add shellcheck as a linter.
Fixes #10162.

Tweaked by tabbott to cover documentation as well.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:32:26 -07:00
Anders Kaseorg d0fb34e8af install: Fix shellcheck warnings.
In scripts/lib/install line 71:
ZULIP_PATH="$(readlink -f $(dirname $0)/../..)"
                          ^-- SC2046: Quote this to prevent word splitting.
                                    ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 105:
mem_kb=$(cat /proc/meminfo | head -n1 | awk '{print $2}')
             ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.

In scripts/lib/install line 141:
apt-get -y dist-upgrade $APT_OPTIONS
                        ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 145:
    $ADDITIONAL_PACKAGES
    ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 254:
    if [ -n "ZULIP_ADMINISTRATOR" ]; then
             ^-- SC2157: Argument to -n is always true due to literal strings.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:26:21 -07:00
Anders Kaseorg 49ae9c1e44 pre-commit: Fix shellcheck warnings.
In tools/pre-commit line 18:
if [ -z "$VIRTUAL_ENV" ] && `which vagrant > /dev/null` && [ -e .vagrant ]; then
                            ^-- SC2092: Remove backticks to avoid executing output.
                            ^-- SC2006: Use $(..) instead of legacy `..`.
                             ^-- SC2230: which is non-standard. Use builtin 'command -v' instead.

In tools/pre-commit line 23:
    ./tools/lint --no-gitlint --force $changed_files || true
                                      ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:26:21 -07:00
Tim Abbott 695d8d0bd1 get_apns_payload: Require a UserProfile object for the recipient.
This is preparatory work for being able to display an unread count
badge on iOS, in which case we need to know who the current user is.
2018-10-18 15:09:18 -07:00
Tim Abbott a4df001cef check_queue_worker_errors: Add support for running unprivileged.
Previously, this script needed access to Django settings, which in
turn required access to /etc/zulip/zulip-secrets.conf.  Since that
isn't world-readable, this meant that this couldn't run as an
unprivileged `nagios` user.

Fix that by just hardcoding the appropriate path under /var/log/.
2018-10-18 15:03:17 -07:00
Rishi Gupta 690705858a billing: Update test_upgrade_where_..._fails_at_first to use @mock_stripe. 2018-10-18 11:16:00 -07:00
Rishi Gupta 03d77bc6af billing: Replace instance.save() Stripe API calls with class.save(instance).
This will make it easier to mock the calls in our new stripe mocking
framework. I believe the two forms are equivalent, assuming the Stripe
Python bindings aren't doing anything crazy. And if not, well hopefully our
new testing framework will catch it :).

[Idea originally from Vishnu KS.]
2018-10-18 11:16:00 -07:00
Rishi Gupta 80c482428d billing: Update test_upgrade_with_outdated_seat_count to use @mock_stripe. 2018-10-18 10:08:40 -07:00
Rishi Gupta 509707007c billing: Update test_billing_page_permissions to use @mock_stripe. 2018-10-18 09:49:50 -07:00
Rishi Gupta c4bad258f7 billing: Import from mock in test_stripe.py.
The string 'mock' appears over 200 times in this file. This removes about 80
of them.
2018-10-17 21:06:09 -07:00
Vishnu Ks 50de59102d billing: Add system to generate fixture data from the billing tests.
This will improve both the maintainability and accuracy of the fixture
data. It also makes it less scary to upgrade Stripe API versions.

[With significant changes by Rishi Gupta.]
2018-10-17 20:35:42 -07:00
Anders Kaseorg fc24d2e147 install-aws-server: Fix shellcheck warnings.
In tools/setup/install-aws-server line 25:
zulip_root=${ZULIP_ROOT:-$HOME/zulip}
^-- SC2034: zulip_root appears unused. Verify use (or export if used externally).

In tools/setup/install-aws-server line 40:
if [ -n "$zulip_confdir" ]; then
         ^-- SC2154: zulip_confdir is referenced but not assigned.

In tools/setup/install-aws-server line 55:
VIRTUALENV_NEEDED=$(if $(echo "$type" | grep -q app_frontend); then echo -n yes; else echo -n no; fi)
                       ^-- SC2091: Remove surrounding $() to avoid executing output.

In tools/setup/install-aws-server line 60:
SSH_OPTS=(-o HostKeyAlgorithms=ssh-rsa)
             ^-- SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

In tools/setup/install-aws-server line 69:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

In tools/setup/install-aws-server line 86:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:51:41 -07:00
Anders Kaseorg b15063155c postgres-init-dev-db: Fix shellcheck warnings.
In tools/setup/postgres-init-dev-db line 10:
ROOT_POSTGRES="sudo -i -u "$DEFAULT_USER" psql"
                           ^-- SC2027: The surrounding quotes actually unquote this. Remove or escape them.

In tools/setup/postgres-init-dev-db line 46:
    echo 'ERROR: Try `sudo service postgresql start`?'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

In tools/setup/postgres-init-dev-db line 64:
PGPASS_ESCAPED_PREFIX="*:\*:\*:$USERNAME:"
                         ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".
                            ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:48:57 -07:00
Anders Kaseorg 310027f970 terminate-psql-sessions: Fix shellcheck warnings.
In scripts/setup/terminate-psql-sessions line 16:
major=$(echo "$version" | cut -d. -f1,2)
^-- SC2034: major appears unused. Verify use (or export if used externally).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:48:57 -07:00
Anders Kaseorg 7f35ad916e django-template-graph: Fix shellcheck warnings.
In tools/django-template-graph line 10:
    for t in $(find -name '*.html' -printf '%P\n'); do
             ^-- SC2044: For loops over find output are fragile. Use find -exec or a while read loop.
               ^-- SC2185: Some finds don't have a default path. Specify '.' explicitly.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg c63d852b55 deploy-branch: Fix shellcheck warnings.
In tools/deploy-branch line 17:
[ $? -ne 0 ] && error_out "Unknown branch: $branch"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 23:
    if [ $? -eq 0 ]; then
         ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 35:
[ $? -ne 0 ] && error_out "Rebase onto origin/master failed"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 39:
[ $? -ne 0 ] && error_out "Push of master to origin/master failed"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg 4a8074000c commit-msg: Fix shellcheck warnings.
In tools/commit-msg line 9:
if [ $(grep '^[^#]' .git/COMMIT_EDITMSG --count) -ne 0 ]; then
     ^-- SC2046: Quote this to prevent word splitting.

In tools/commit-msg line 10:
    lint_cmd="cd ~/zulip && cat \"$1\" | python -m gitlint.cli"
              ^-- SC2089: Quotes/backslashes will be treated literally. Use an array.

In tools/commit-msg line 11:
    if [ -z "$VIRTUAL_ENV" ] && `which vagrant > /dev/null` && [ -e .vagrant ]; then
                                ^-- SC2092: Remove backticks to avoid executing output.
                                ^-- SC2006: Use $(..) instead of legacy `..`.
                                 ^-- SC2230: which is non-standard. Use builtin 'command -v' instead.

In tools/commit-msg line 14:
        $lint_cmd
        ^-- SC2090: Quotes/backslashes in this variable will not be respected.

In tools/commit-msg line 17:
    if [ $? -ne 0 ]; then
         ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg 8b38f61176 clean-branches: Fix shellcheck warnings.
In tools/clean-branches line 33:
                echo -n "Deleting local branch $(echo "$ref" | sed 's!^refs/heads/!!')"
                                                 ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/clean-branches line 41:
                echo -n "Deleting local branch $(echo "$ref" | sed 's!^refs/heads/!!')"
                                                 ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/clean-branches line 49:
                remote_name="$(echo "$ref" | sed 's!^refs/remotes/origin/!!')"
                               ^-- SC2001: See if you can use ${variable//search/replace} instead.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg b5bfe77844 build-release-tarball: Fix shellcheck warnings.
In tools/build-release-tarball line 50:
for i in `cat "$TMPDIR/$prefix/tools/release-tarball-exclude.txt"`; do
         ^-- SC2013: To read lines rather than words, pipe/redirect to a 'while read' loop.
         ^-- SC2006: Use $(..) instead of legacy `..`.

In tools/build-release-tarball line 51:
    rm -r --interactive=never "$TMPDIR/$prefix/$i";
                              ^-- SC2115: Use "${var:?}" to ensure this never expands to / .

In tools/build-release-tarball line 97:
    echo; echo -ne "\033[33mRunning update-prod-static failed. "
                    ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/build-release-tarball line 98:
    echo -e "Check $TMPDIR/update-prod-static.log for more information.\033[0m"
                                                                       ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg a5280d7d02 build-docs: Fix shellcheck warnings.
In tools/build-docs line 3:
cd "$(dirname "$0")"/../docs
^-- SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Tim Abbott b47b5c5309 docs: Update changelog to cover everything in latest master. 2018-10-17 17:32:24 -07:00
Tim Abbott fde03feed9 api docs: Fix titles for add/remove linkifiers docs. 2018-10-17 15:35:53 -07:00
Eeshan Garg b31bbbe649 api_docs: Use tabbed_sections extension in new API docs. 2018-10-17 15:32:35 -07:00
Eeshan Garg 5275d49f05 api_docs: Properly escape arguments passed to curl.
Fixes #8484.
2018-10-17 15:32:35 -07:00
Tim Abbott a34b79a3f4 python: Avoid importing the mock module in production.
These lazy imports save a significant amount of time on Zulip's core
import process, because mock imports pbr, which in turn import
pkgresources, which is in turn incredibly slow to import.

Fixes part of #9953.
2018-10-17 15:28:48 -07:00
Steve Howell ff61c56f47 hipchat import: Add NotificationMessage support. 2018-10-17 12:11:08 -07:00
Tim Abbott f9b6eeb488 import: Migrate from json to ujson for better perf.
We expect to get better memory performace from
ujson than json.

We also do a better job of closing file handles.

This likely fixes #10377.
2018-10-17 12:11:08 -07:00
Tim Abbott 2eebacf2dc push_notifications: Lazily import APNS libraries.
The APNS client libraries (especially the hyper.http20 one) were
determined via profiling to take significant time during the import
process, so we move them to be lazily imported in order to optimize
the overall Zulip import process.  This save up to about 100ms in
import time.

These libraries are only used in certain Django processes inside
zulipchat.com, and so are unnecessary both in development as well as
for self-hosted Zulip servers.
2018-10-17 11:59:33 -07:00
Tim Abbott ec9f6702d8 manage.py: Sabotage importing pika.adapters.twisted_connection.
This is a performance optimization; see the comment.  This fixes part
of #9953.

Eventually, we should do the same thing for importing Tornado as well,
but it's less important because Tornado is a much smaller library.
2018-10-17 11:50:08 -07:00
Tim Abbott 704967faa4 email_mirror: Don't import talon unless we're using it.
Talon is an expensive import; on my system, deferring this import
saves 28ms on the import time for Zulip.
2018-10-17 11:25:38 -07:00
Tim Abbott d58e5b8c16 requirements: Use a forked version of libthumbor for performance.
This optimizes the import performance of Zulip by using a fork of
pkgresources.  The fork just has a single commit, which replaces
libthumbor's use of pkgresources (which is really slow to import) with
a direct query to the version number.

This fixes part of #9953.
2018-10-17 11:21:29 -07:00
Rishi Gupta d76f728dda api docs: Rename webhook-walkthrough to incoming-webhooks-walkthrough. 2018-10-16 21:07:00 -07:00
Rishi Gupta 6a352ab249 api docs: Add script and plugin integrations to integrations-overview. 2018-10-16 21:07:00 -07:00
Rishi Gupta df20f1a47f api docs: Rename integration-guide.md to non-webhook-integrations.md. 2018-10-16 21:07:00 -07:00
Rishi Gupta cd81b0766a api docs: Move references to integration-guide to integrations-overview. 2018-10-16 21:06:59 -07:00
Rishi Gupta a8e1225a6b api docs: Update integration-guide. 2018-10-16 21:06:59 -07:00
Rishi Gupta c3eb922559 api docs: Move general advice to incoming webhooks guide.
One of the bullets does reference APIs, which aren't relevant to incoming
webhooks, but decided to leave it as is.
2018-10-16 21:06:59 -07:00
Rishi Gupta decc5352aa api docs: Update webhook-walkthrough. 2018-10-16 21:06:59 -07:00
Rishi Gupta 47cddf4acf api docs: Update incoming-webhooks-overview.md. 2018-10-16 21:06:59 -07:00
Rishi Gupta 4abce1783f api docs: Restructure integrations section and add an overview. 2018-10-16 21:06:59 -07:00
dipu989 ef2345414d settings: Change username label. 2018-10-16 17:09:21 -07:00
dipu989 ea5bb4a150 settings: Interchange Username and Bot Type positions. 2018-10-16 17:09:20 -07:00
Steve Howell fd22687cd2 refactor: Split out buddy_list_conf().
This moves us closer to having a generic widget that
we can use for other things like stream settings.
2018-10-16 16:53:47 -07:00