Commit Graph

24170 Commits

Author SHA1 Message Date
Tim Abbott afda05adc4 rate limiting: Increase default rate limits.
It's too easy to go over the rate limits when using the webapp.

The correct fix for this probably involves some changes to which
routes get covered by what sort of rate limit, but for now, just
increase the limits.
2018-02-09 11:55:08 -08:00
Sumana Harihareswara ea0fe90367 webhooks/beeminder/doc.md: Use better macros. 2018-02-09 11:53:32 -08:00
Tim Abbott 77addc5456 bugdown: Fix handling of ultra-long renderings.
If some bug in Bugdown results in a rendered message content that is
bigger than twice the message size, we now just throw an exception
from Bugdown.  This is considerably better than the old behavior,
which might result in an enormous message being placed in the database
(potentially, bigger than the 1MB limit to store in memcached), which
would in turn result in tragic consequences.

This fixes #8322, in that it prevents the super bad outcome seen there
(where basically Zulip became unusable for everyone on the stream
where the message is posted).  Now, the failure mode is just the
message failing to send.  Still not ideal (and requires further work
on the URL embed feature), but not a minor problem, not a major one.
2018-02-09 10:57:55 -08:00
Shubham Dhama 9cd7f59c9b recent senders: Use message ids instead of timestamps for sorting.
This matches the convention that we use everywhere else in Zulip.

Fixes: #5956.
2018-02-08 18:39:10 -08:00
Vishnu Ks ebe16e064e README: Include CircleCI build status badge.
For now, we still need the Travis badge, since Travis is where we test the 
production installation process.  But ideally, we'll end up removing that too.
2018-02-08 18:24:43 -08:00
Greg Price 7df29e7a7c puppet: Only use those "modern" options when on xenial.
On trusty, we of course have an older version -- 1.4.14 -- and it is
not so modern, so this just gives an error.
2018-02-08 18:11:52 -08:00
Eeshan Garg 3853577486 api tests: Add example/tests for unsubscribing other users.
This commit adds tests (and thus, an extra code example) for
unsubscribing another user from a particular stream by passing in
the `principals` argument to client.remove_subscriptions. The
ability to pass in `principals` was added in the latest release
of the zulip API PyPI package.
2018-02-08 17:58:41 -08:00
Eeshan Garg ac5c355a60 api/delete-queue: Make fixtures/examples testable.
api/delete-queue now uses the Markdown extension in
bugdown/api_code_examples to generate code examples and fixtures
from tests.
2018-02-08 17:58:41 -08:00
Eeshan Garg 344cf2c8ee api_tests: Add tests for client.register_queue. 2018-02-08 17:58:41 -08:00
Eeshan Garg 66ca0c725f api/get-all-users: Make fixtures/examples testable.
api/get-all-users now uses the Markdown extension in
bugdown/api_code_examples to generate code examples and fixtures
from tests.
2018-02-08 17:58:41 -08:00
Eeshan Garg 929724e5e7 api docs: Add page for common error payloads.
We now have a separate page for common error payloads, for example,
the payload for when the client's API key is invalid. All error
payloads that are presented on this page will be tested similarly
to our other non-error sample fixtures.
2018-02-08 17:58:41 -08:00
Eeshan Garg 037c891fcf webhooks/delighted: Reformat documentation steps into numbered list.
This is a part of our efforts to migrate our webhook docs to have
a more structured numbered-list format similar to our /help docs.
2018-02-08 17:45:56 -08:00
Eeshan Garg 69f248aaf9 webhooks/codeship: Reformat documentation steps into numbered list.
This is a part of our efforts to migrate our webhook docs to have
a more structured numbered-list format similar to our /help docs.
2018-02-08 17:45:56 -08:00
Eeshan Garg 06e86ce50d webhooks/bitbucket2: Reformat documentation steps into numbered list.
This is a part of our efforts to migrate our webhook docs to have
a more structured numbered-list format similar to our /help docs.
2018-02-08 17:45:56 -08:00
Eeshan Garg 5e1a193e76 webhooks/circleci: Reformat documentation steps into numbered list.
This is a part of our efforts to migrate our webhook docs to have
a more structured numbered-list format similar to our /help docs.
2018-02-08 17:45:56 -08:00
Eeshan Garg af683800fd webhooks/basecamp: Reformat documentation steps into numbered list.
This is a part of our efforts to migrate our webhook docs to have
a more structured numbered-list format similar to our /help
docs.
2018-02-08 17:45:56 -08:00
Eeshan Garg ad6dae1bf5 webhooks/appfollow: Reformat documentation steps into numbered list.
This is a part of our efforts to migrate our webhook docs to have
a more structured numbered-list format similar to our /help docs.
2018-02-08 17:45:56 -08:00
Eeshan Garg 2733540492 webhooks/airbrake: Reformat documentation steps into numbered list.
This is a part of our efforts to migrate our webhook docs to have
a more structured numbered-list format similar to our /help docs.
2018-02-08 17:45:56 -08:00
Greg Price 86590dfdbe test-install: Add command destroy-all to clean up test containers.
This is just the one-liner I've been keeping in my shell history,
cleaned up a bit (newlines!) and with 28 lines of CLI boilerplate
added in front.
2018-02-08 17:29:41 -08:00
Greg Price 6e633f8e2f install: Use readlink -f rather than realpath.
It does exactly the same thing, though the name is less transparent; and
it simplifies the script by avoiding an extra, early `apt-get install`.
2018-02-08 17:22:02 -08:00
Greg Price 8d86a6e331 install: Slightly clean up another couple of bits of style. 2018-02-08 17:14:28 -08:00
Greg Price 218b653930 install: Clean up organization of option parsing.
The parsing of options from environment variables belongs right next
to the normal kind; and `show_help` was unnecessary indirection.
2018-02-08 17:12:36 -08:00
Greg Price 08135e3666 install: Don't try to chown a self-signed cert to zulip.
The zulip user has no need to see this file; it's used by nginx.
And when we set up the cert early in install, there's no zulip user
yet anyway, so this fails.
2018-02-08 17:08:16 -08:00
Greg Price f68c485a86 install: Clean up some shell style: use here-docs, and compact getopt. 2018-02-08 17:05:13 -08:00
Greg Price 8f4b7f6e1e install: On --express, check up front for --hostname and --email. 2018-02-08 16:34:49 -08:00
Greg Price 31ec9a4627 install: Implement --self-signed-cert earlier.
Otherwise when puppet tries to install and start nginx, that can
give an error.
2018-02-08 16:34:49 -08:00
Greg Price 4c5326ce85 test-install: Factor out booted-yet-p polling loop, use in prepare-base.
Otherwise prepare-base is likely to fail when first run (but then
succeed when rerun, because the container is left running), because
the container isn't up yet when we try to operate in it.

Also clean up the placement of `set -e` vs `set -x`.
2018-02-08 16:34:49 -08:00
Greg Price fc9970e561 test-install: Add xenial support. 2018-02-08 16:34:49 -08:00
Greg Price 23e6a2e579 puppet: Update memcached config to turn on this decade's technology.
We've been running this change on zulipchat.com for a couple of months
now.  Before then, we used to regularly get exceptions like this:

     File "./zerver/views/messages.py", line 749, in get_messages_backend
       setter=stringify_message_dict)
     File "./zerver/lib/cache.py", line 275, in generic_bulk_cached_fetch
       cache_set_many(items_for_remote_cache)
     File "./zerver/lib/cache.py", line 215, in cache_set_many
       get_cache_backend(cache_name).set_many(items, timeout=timeout)
     File "/home/zulip/deployments/2017-09-28-21-04-12/zulip-py3-venv/lib/python3.5/site-packages/django/core/cache/backends/memcached.py", line 150, in set_many
       self._cache.set_multi(safe_data, self.get_backend_timeout(timeout))
   pylibmc.Error: error 48 from memcached_set_multi

This error means memcached was unable to find space for the new value.
You might think that because memcached provides an LRU cache, this
shouldn't happen because it would just evict something... but in fact
  * memcached splits its data into "slabs" by object size, and
  * until recently, once a 1MiB "chunk" is allocated to a given "slab"
    i.e. size class, it wouldn't be reclaimed to allocate to another.

So once the cache has been filled up with objects of some distribution
of sizes, if some objects come in that would go in a different size
class, we have no chunks for that size class / slab, and can't get one.
And that's exactly what was happening on zulipchat.com.

Useful background can be found in
  https://github.com/memcached/memcached/wiki/ServerMaint#slab-imbalance
  https://github.com/memcached/memcached/wiki/ReleaseNotes1411
  https://github.com/memcached/memcached/wiki/ReleaseNotes1425
  https://github.com/memcached/memcached/wiki/ReleaseNotes150
We're already running v1.4.25, which provides an "automover" that should
be well equipped to fix this; v1.5.0 turns it on by default.

With this commit, adopt the "modern start line" recommended in the
release notes for our v1.4.25, including turning on the automover.
2018-02-08 16:34:49 -08:00
Eeshan Garg 4414f2b24b webhooks/beeminder: Use create-a-bot.md macro. 2018-02-08 16:22:55 -08:00
Eeshan Garg ab700fb662 markdown: Add a macro with instructions for creating a bot. 2018-02-08 16:22:55 -08:00
Rhea Parekh 48640fd28f slack importer: Suppress logger output from the unit tests. 2018-02-08 16:21:35 -08:00
Vishnu Ks 680983d10e vagrantfile: Workaround for the atlas.hashicorp.com redirect removal.
This workaround is only for new users. Users who are/were using old
versions of Vagrant has to manually change the value of box
metadata_url. See https://github.com/hashicorp/vagrant/issues/9442
2018-02-08 13:54:19 -08:00
Tim Abbott 9249454518 update-authors-json: Fix handling deleted GitHub accounts.
Apparently, we've now had the first time one of our contributors had
their account deleted (at least, the author page for the contributor
who has 21 commits in python-zulip-api now 404s).
2018-02-08 12:17:20 -08:00
Greg Price 9e70026741 settings: Slightly clarify LDAP setup docs.
There's kind of a lot of prose in this LDAP section; and empirically
this bit about zulip-secrets.conf is easy to miss.
2018-02-08 10:05:25 -08:00
Robert Hönig 537203102a api tests: Update remove-subscriptions fixture.
The remove-subscriptions API client response
changed with python-zulip-api:67cdb6f
2018-02-08 09:37:41 -08:00
Brock Whittaker e82564c82c dark-mode: Fix bug with open graph previews.
This fixes a bug where the open graph preview bottom fade is dark
rather than white when not in dark mode, which results in a heavy
dark faded line at the bottom of the description.
2018-02-08 09:22:58 -08:00
Eeshan Garg 61e4fc3b74 api/remove-subscriptions: Make fixtures/examples testable.
api/remove-subscriptions now uses the Markdown extension in
bugdown/api_code_examples to generate code example and fixtures
from tests.
2018-02-07 15:58:57 -08:00
Eeshan Garg f4cb5e70ed api/add-subscriptions: Make fixtures/examples testable.
api/add-subscriptions now uses the Markdown extension in
bugdown/api_code_examples to generate code example and fixtures
from tests.
2018-02-07 15:58:57 -08:00
Eeshan Garg 09721e9ec3 bugdown/api_code_examples: Support multiple fixtures per doc.
This commit adds support for passing in an argument to the macro
"call" to explicitly specify a fixture to render, like so:

{generate_code_example|doc_name|fixture(stream_message_with_args)}
2018-02-07 15:58:57 -08:00
Eeshan Garg 7ae704d1bb api/get-all-streams: Render multiple examples from single test method.
api/get-all-streams - make use of the api_code_example Markdown extension's
feature of recursively extracting multiple code examples from a single
test method.
2018-02-07 15:58:57 -08:00
Eeshan Garg 0cca39d1fc bugdown/api_code_examples: Support multiple examples in one test.
This commit adds support for recursively extracting multiple code
examples from a single test method.
2018-02-07 15:58:57 -08:00
Eeshan Garg ffe930d50d bugdown/api_generate_examples: Replace 'method' with 'example'.
To generate a code exammple,

{generate_code_example|<api_doc_md>|example} sounds better and
more intuitive than,

{generate_code_example|<api_doc_md>|method}
2018-02-07 15:58:57 -08:00
Greg Price ccb8c79edb test_docs: Fix a now-brittle test.
The count of integrations is automatically computed now, so this will
change every time we add 10 more.  Just stop asserting on the number.

Thanks to @hackerkid for spotting the issue.
2018-02-07 13:01:42 -08:00
Josh Mandel 15eaafa50f Add underscores to thumbs_up and thumbs_down emoji 2018-02-07 11:55:02 -08:00
Ricky db152da7c9 webhooks/beeminder/doc.md: Add instructions for creating a bot. 2018-02-07 10:03:31 -08:00
Ricky 77760d9f6c webhooks/beeminder: Update sample message screenshot. 2018-02-07 10:03:31 -08:00
Ricky 1303248b0b integration: Add statuspage.io webhook integration. 2018-02-07 09:59:12 -08:00
Eeshan Garg d7903d25e9 pypi_packages: Upgrade to release 0.4.1. 2018-02-07 09:52:51 -08:00
Greg Price c80802ff1e bots: Clean up create_realm_internal_bots.
This code duplicated the code in setup_realm_internal_bots, with some
added logic to avoid trying to create the same bot twice.  That logic
was buggy so that it would never work at all -- it subtracted a set of
UserProfile objects from a set of email strings -- so it looked like
the command might blow up when run after the users already existed.

In fact, the buggy logic wasn't necessary, because the work the
command does after it is idempotent -- in particular `create_users`,
within its subroutine `bulk_create_users`, already filters out users
that already exist.  So just cut the buggy stuff out, deduplicate the
rest with `setup_realm_internal_bots`, and document that invariant on
the latter.

While we're here, in the common case bail early without doing any
per-realm work in Python, since we're running this on every upgrade.
2018-02-06 18:59:57 -08:00